Method for predicting change of anatomical structure in radiotherapy fraction of radiotherapy patient

Abstract

The invention discloses a method for predicting change of an anatomical structure in a radiotherapy fractional interval of a radiotherapy patient, and belongs to the technical field of medical treatment. Comprising the steps of data collection, radiomics feature extraction and analysis, deep learning model training and optimization, test set verification model, clinical verification set test, clinical verification set calculation and analysis, model final output result prediction and the like. According to the method, on the basis of MR images, positioning CT images and radiotherapy CBCT images, a prediction model of anatomical structure changes during radiotherapy is constructed and optimized in a radiomics feature analysis and deep learning mode, and on the basis of individual features of a specific patient, anatomical images of the specific patient in future fractional treatment are predicted; a basis is provided for adopting a more active and optimized self-adaptive radiotherapy triggering mechanism, and a more accurate and timely personalized radiotherapy scheme is provided for a patient.

Description

Technical Field

[0001] The invention belongs to the field of medical technology, and in particular relates to a method for predicting changes in in vivo anatomical structures of radiotherapy patients between radiotherapy fractionations. Background Art

[0002] At present, the conventional fractionation dose of radiotherapy is PTV: 2Gy / time for 25 times, and the local boost dose of parametrial lymph nodes PGTVnd: 2Gy / time for 5 times, with a total course of 30 times, and radiotherapy is completed in six weeks. The standard radiotherapy process mainly includes two stages. (1) Preparation stage: First, the patient's three-dimensional simulation positioning CT image is obtained through a large-caliber CT; then, the radiotherapy doctor outlines the tumor target area and the organ at risk in the CT image; finally, the medical physicist designs the radiotherapy plan on the CT image for the patient's treatment. (2) Treatment stage: First, the patient is positioned according to the position during the simulation positioning; then, the patient's online cone beam CT (CBCT) image is acquired to correct the positioning error; finally, the designed radiotherapy plan is executed. There are some problems in this conventional process that have not been solved, which affect the accuracy of treatment implementation. The first problem is that during fractionated treatment, the patient's anatomical structure within the irradiation range may shift relative to the simulated positioning due to factors such as weight changes or tumor volume changes. Simple repositioning is not enough to compensate for the non-rigid deformation of the anatomical structure. Implementation of the original radiotherapy plan will result in insufficient target dose coverage or increased irradiation dose to endangered organs.

[0003] Displacement and deformation of internal anatomical structures, organ movement during irradiation, and changes in the physical and biological properties of tumors and normal tissues during the entire course of treatment will all cause redistribution of radiotherapy radiation doses. Because both the incident direction of the radiation field and the steep dose gradient require accurate patient models, when the conformality of the radiation field increases, organ movement can cause the dose distribution to become blurred, thereby affecting the accuracy of the dose distribution. Therefore, for the patient model created during the planning process, the positioning uncertainty and organ motion error of the entire treatment process must be considered.

[0004] With the advancement of technology and the deepening of clinical research, adaptive radiotherapy will play an increasingly important role in treatment. Researchers at home and abroad are exploring the integration of adaptive radiotherapy with other technologies such as artificial intelligence and multimodal imaging to further improve the automation of treatment and simplify the radiotherapy workflow. Future research needs to further verify the clinical effect of adaptive radiotherapy and explore its optimal application strategy in treatment. The challenges currently faced mainly include: how to accurately define moving and changing tumor targets; how to adjust the treatment plan based on the patient's real-time images to adapt to changes in tumors and normal tissues; how to integrate multiple advanced technologies, such as MRI, CT, PET-CT, etc., to improve the accuracy and personalization of adaptive radiotherapy.

[0005] Anatomical changes may occur at any time during the treatment of cancer patients, ranging from a few seconds to a few weeks. These changes may affect tumors and normal organs, such as cardiac and respiratory movements, bladder filling, prostate rotation, rectal filling, lung re-ventilation, and cervical shape changes. ART has developed a variety of adaptation strategies based on different anatomical changes, including average anatomical models, library of plans, triggered adaptation, and scheduled adaptation. Accurately predicting possible anatomical changes can effectively provide a basis for the formulation of adaptive radiotherapy strategies. There have been relevant studies on predicting anatomical changes in head and neck tumors through techniques such as deformable registration and deep learning, but the prediction of anatomical changes during radiotherapy has not been reported.

[0006] To this end, we propose a method for predicting the changes in anatomical structures in radiotherapy patients between radiotherapy fractions to solve the problems existing in the existing technology, provide basic data and clinical basis for the precise adaptive radiotherapy plan, and enable patients to receive individualized and precise treatment. Summary of the invention

[0007] The purpose of the present invention is to provide a method for predicting changes in anatomical structures in a radiotherapy patient between radiotherapy fractions, so as to solve the problem of predicting anatomical changes during radiotherapy in the prior art proposed in the above background technology.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions comprises the following steps:

[0010] Collection and management of imaging data: Collect the patient's imaging data before and after radiotherapy and establish a standardized data set;

[0011] High-throughput imaging features extraction and analysis: Based on medical knowledge, imaging features related to radiotherapy response are selected, including morphology, texture, and signal intensity distribution features;

[0012] Build a deep learning network model based on multimodal data to predict anatomical changes during radiotherapy, design and train a deep learning model to analyze imaging features and predict anatomical changes;

[0013] Evaluation and optimization of model performance: Evaluate performance indicators such as model accuracy and optimize the model.

[0014] Preferably, the image data acquisition comprises the following steps:

[0015] Diagnostic multimodal MR images, localization CT images, CBCT images at days 1-3 and the first time in weeks 2-6, and radiotherapy plans were collected;

[0016] Preprocess the CBCT images and convert them into pseudo-CT;

[0017] The deep learning network is trained to predict the CBCT image at a certain time during the radiotherapy course, representing its anatomical image, and output the contour results of the target area and organs at risk on the CBCT.

[0018] Preferably, the high-throughput imaging genomics feature extraction and analysis comprises the following steps:

[0019] Perform high-throughput imaging genomics feature extraction and screening on multimodal MR images and positioning CT images;

[0020] For the deep learning feature branch, multiple continuous stacks of 3D convolutional neural networks are used to screen key features;

[0021] Multiple serial stacks and radiomics feature screening results were input into a multimodal hierarchical network.

[0022] Preferably, the deep learning model training and optimization comprises the following steps:

[0023] Using pre-radiotherapy positioning CT, multimodal MR images, and the acquired CBCT images of each treatment fraction as input, the CBCT images and target volume and risk organ structures of each future fraction are predicted;

[0024] Use deep learning-based networks to build multi-task models;

[0025] CycelGAN is used to achieve end-to-end image prediction, and an improved segmentation model based on Unet is used to achieve contour prediction of tumors and endangered organs. The Encoder of the synthetic CBCT in cycelGAN shares a structure with the Encoder of the segmentation network. The Lossseg of the segmentation network is added to the LosscycelGAN of the cycelGAN network to form the Loss. The generation network and the segmentation network are trained together.

[0026] Preferably, the quantitative evaluation indicators of anatomical images include MAE, RMSE, SSIM, and PSNR.

[0027] Preferably, the quantitative evaluation indicators of the anatomical structure include the Dyss similarity coefficient and the Hausdorff distance.

[0028] Preferably, the anatomical structure change prediction comprises three steps of feature extraction, multimodal feature aggregation and feature re-measurement, wherein:

[0029] The feature extraction performs high-throughput imaging genomics feature extraction on multimodal MR images and positioning CT images. For deep learning feature branches, multiple continuous stacks of 3D convolutional neural networks are used to screen key features using feature dimensionality reduction methods such as PCA and Lasso, and the key features are input into the multimodal hierarchical network.

[0030] Preferably, the multimodal feature aggregation uses two independent processing branches to perform feature learning;

[0031] Among them, the stacked convolutional layers and downsampling layers perform feature learning layer by layer, and in this process, the features are kept in three-dimensional form;

[0032] For the imaging omics feature processing branch, a one-dimensional fully connected neural network is used to perform feature learning layer by layer. During this process, the features are kept in one-dimensional form.

[0033] Preferably, the feature weighting utilizes aggregated multimodal features to learn the weights when fusing features, indicating the importance of features of two different modalities to the final anatomical image, thereby performing weighted measurement on the features of two different modalities.

[0034] Technical effects and advantages of the present invention: The method for predicting changes in the in vivo anatomical structure of a radiotherapy patient between radiotherapy fractions proposed by the present invention has the following advantages compared with the prior art:

[0035] 1. Based on MR images, positioning CT images, and radiotherapy CBCT images, the present invention uses radiomics feature analysis and deep learning to build and optimize a prediction model for anatomical structure changes during radiotherapy. Based on the individual characteristics of specific patients, the present invention predicts their anatomical images in future fractionated treatments, providing a basis for adopting a more active and optimized triggering mechanism for adaptive radiotherapy, and providing patients with more accurate and timely personalized radiotherapy plans.

[0036] 2. The present invention adopts a method of extracting and fusing image features of different modalities, which can enable the training network to effectively extract more biological features representing the imaging genomics information, thereby making the model applicable to images of different modalities, thereby making the model more widely used and the prediction results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flow chart of a method for predicting changes in in vivo anatomical structures of radiotherapy patients between radiotherapy fractions according to the present invention;

[0038] Figure 2 The present invention is a flowchart of a method for predicting the level of future anatomical structure changes using a multimodal hierarchical network in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] See also Figure 1-Figure 2 The present invention discloses a method for predicting changes in the in vivo anatomical structure of a radiotherapy patient between radiotherapy fractionation, comprising the following steps:

[0041] S1. Image data collection and management: Collect the patient's image data before and after radiotherapy and establish a standardized data set;

[0042] Specifically, in this embodiment, deep learning is performed based on diagnostic MR, positioning CT, and each fractionated CBCT image to predict the anatomical image of the patient during the radiotherapy course. The main steps include: collecting data, including diagnostic multimodal MR images, positioning CT images, CBCT images of days 1-3 and the first time in weeks 2-6, and radiotherapy plans; preprocessing the CBCT images and converting them into pseudo-CT; training a deep learning network to predict the CBCT image at a certain time during the radiotherapy course, representing its anatomical image, and outputting the contour results of the target area and organs at risk on the CBCT. The inclusion conditions of the data are determined according to the discussion of the expert group, the included data are checked and confirmed, and the quality of the data is evaluated with reference to the inclusion criteria and sample set. If there are contours that do not meet the standards, they need to be modified to make them consistent with the standards to form a standard database. Check the patient's medical records, patient images, and radiotherapy plans to ensure that they meet clinical requirements.

[0043] In addition, in this embodiment, the expert group is responsible for data collection and inspection, standard setting and result evaluation.

[0044] S2. High-throughput imaging omics feature extraction and analysis: Based on medical knowledge, imaging omics features related to radiotherapy response were selected, including morphology, texture, and signal intensity distribution features;

[0045] Specifically, the anatomical image and anatomical structure prediction uses the pre-radiotherapy positioning CT, multimodal MR images, and the CBCT of each treatment fraction that has been collected as input, and predicts the CBCT images and target area and risk organ structures of each future fraction.

[0046] A multi-task model is constructed using a deep learning-based network. In this model, each image is input as a separate channel, and modules such as feature cross-fusion are used after multiple input channels to ensure that information of different image features is fully extracted and fused at different levels.

[0047] For the output of the model, the CBCT and contour results in future fractions are predicted as dual-path outputs.

[0048] CycelGAN is used to achieve end-to-end image prediction, and the improved segmentation model based on Unet is used to achieve the contour prediction of tumors and organs at risk. For the shared structure of the Encoder of the synthetic CBCT in cycelGAN and the Encoder of the segmentation network, the Lossseg of the segmentation network is added to the LosscycelGAN of the cycelGAN network to form the Loss, and the generation network and the segmentation network are trained together.

[0049] The multi-input and multi-task architecture model will make full use of image features and also promote the generation of more accurate multi-task output results.

[0050] Furthermore, considering that this embodiment involves many types of data and has the characteristics of multimodal input, this embodiment adopts a multimodal hierarchical network, and the prediction of anatomical structure changes includes three steps: feature extraction, multimodal feature aggregation, and feature re-measurement.

[0051] High-throughput imaging genomics feature extraction is performed for multimodal MR images and positioning CT images. For the deep learning feature branch, multiple continuous stacks of 3D convolutional neural networks are used to screen key features using feature dimensionality reduction methods such as PCA and Lasso, and input them into the multimodal hierarchical network. Since the features extracted by deep learning and imaging genomics features belong to different modalities, in order to learn targeted features separately, two independent processing branches are used for feature learning. The stacked convolutional layers and downsampling layers perform feature learning layer by layer. In this process, the features are kept in a three-dimensional form. For the imaging genomics feature processing branch, a one-dimensional fully connected neural network (multi-layer perceptron, MLP) is used to perform feature learning layer by layer. In this process, the features are kept in a one-dimensional form.

[0052] In order to accurately predict the anatomical structure of patients during radiotherapy, it is necessary to effectively integrate the three-dimensional form features and one-dimensional form features extracted by the above two branches, which requires dimensional matching of the features of the two different modalities. To this end, the three-dimensional form features are first subjected to a spatial global average pooling operation, and the feature vectors generated after pooling have a scale of 1×d. The imaging omics features obtained by processing with a one-dimensional fully connected neural network are finally transformed into a 1×d feature vector.

[0053] After that, the 1×d feature vectors generated by the two modalities are fused. The operation flow of multimodal feature fusion is shown in the figure above. Its structure consists of two parts, one is the feature aggregation module, and the other is the feature weight measurement module.

[0054] First, the two 1×d feature vectors generated by the above two features are passed through two linear connection layers respectively and the vector size is kept unchanged, and the feature size is still 1×d. After that, the two generated feature vectors are processed by layer normalization to speed up the training of the model and avoid model overfitting. The two feature vectors after layer normalization are stacked, and the feature vector dimension is transformed to 2×d.

[0055] Afterwards, the weights of the features when fusion is learned by using the above-mentioned aggregated multimodal features to indicate the importance of the features of the two different modalities to the final anatomical image, so as to perform weighted measurement on the features of the two different modalities. Specifically, the above-mentioned aggregated feature vectors are processed continuously by the linear connection layer, tanh activation function, and linear connection layer, and the softmax layer converts the features into weight vectors with a sum of 1 and a dimension of 2×1, indicating the weights corresponding to the features of different modalities. After the above-mentioned weight vectors are obtained, the full convolution module generates the final CBCT image and the corresponding structural image.

[0056] S3. Build a deep learning network model based on multimodal data to predict anatomical changes during radiotherapy, design and train a deep learning model to analyze imaging features and predict anatomical changes;

[0057] Specifically, before building the model, determine the data set first, and divide all the image data into training set, validation set, and test set in a ratio of 8:1:1. Analyze the clinical characteristics of the patients in the three data sets, such as age, gender, clinical stage, image acquisition, etc., to ensure that there are no significant differences in the clinical characteristics of the three data sets. Analyze the size, resolution, layer thickness and other indicators of the input data to determine the data preprocessing method. Design the network model architecture, input the preprocessed data images and the corresponding manual labels into the network model, and use the five-fold cross-validation method to train the model. During the training process, tune the hyperparameters in the network model until the validation set achieves the desired effect.

[0058] In order to evaluate the accuracy of anatomical image prediction, a quantitative evaluation is performed on the predicted CBCT of each fraction, and the evaluation indicators include MAE, RMSE, SSIM, and PSNR.

[0059] MAE (Mean Absolute Error): MAE is a measure of the pixel-level difference between the original image and the image to be evaluated. It is the average of the absolute values ​​of all pixel errors. The smaller the MAE, the smaller the difference between the two images and the better the image quality. The definition formula is as follows:

[0060] MAE=1MN∑i=1M∑j=1N∣I(i,j)-K(i,j)∣MAE=MN1∑i=1M;

[0061] ∑j=1N∣I(i,j)-K(i,j)∣;

[0062] Where I is the original image, K is the image to be evaluated, and M and N are the number of rows and columns of the image, respectively.

[0063] RMSE (Root Mean Square Error): RMSE is the square root of the sum of the squares of MAE, which measures the mean square error between the original image and the image to be evaluated. The smaller the RMSE, the better the image quality. The definition formula is as follows:

[0064] RMSE=1MN∑i=1M∑j=1N(I(i,j)-K(i,j))2RMSE=MN1

[0065] ∑i=1M∑j=1N(I(i,j)-K(i,j))2;

[0066] SSIM (Structural Similarity Index): SSIM is a comprehensive index that takes into account image brightness, contrast, and structure, and is used to measure the perceived quality of two images. The SSIM value range is -1 to 1, where 1 means that the two images are exactly the same and 0 means the image quality is the worst. The definition formula is relatively complex, involving comparisons of brightness, contrast, and structure:

[0067] SSIM(x,y)=(2μxμy+c1)(2σxy+c2)(μy2+μy2+c1)(σx2+σy2+c2)SSIM(x,y)=(μx2+μy2+c1)(σx2+σy2+c2)(2μxμy+c1)(2σxy+c2);

[0068] Among them, x and y are two images, μx and μy are the brightness of the images, 2σx2 and 2σy2 are the contrast of the images, σxy is the structure of the images, and 1c1 and 2c2 are small constants to prevent the denominator from being zero.

[0069] PSNR (Peak Signal-to-Noise Ratio): PSNR is an indicator of image quality, which is calculated based on the mean square error (MSE) of the image. PSNR is usually used to evaluate the quality of image reconstruction or image compression. The unit of PSNR is decibel (dB), and the higher the value, the better the image quality. The definition formula is as follows:

[0070]

[0071] Among them, MAXI is the maximum possible pixel value of the image, and MSEMSE is the mean square error.

[0072] In order to evaluate the accuracy of anatomical structure prediction, doctors outline the target area and organs at risk on the real CBCT images and compare them with the predicted anatomical results to compare the consistency of the two. Evaluation indicators include:

[0073] Dice similarity coefficient (Dicesimilaritycoefficient, Dice):

[0074] DSC = (2|Va∩Vb|) / (Va+Vb);

[0075] Among them, Va and Vb represent the volumes of the manually outlined contours based on the CBCT image and the predicted image, respectively. The value of DSC is between 0 and 1, and the closer it is to 1, the higher the similarity between the two.

[0076] Mean minimum distance to agreement (MDA):

[0077] MDA(A,B)=mean[mean_b(inf_a(d(a,b))),mean_a(inf_b(d(b,a)))];

[0078] Where A and B represent the manually outlined point sets on the CBCT image and the predicted image, respectively. The closer the MDA value is to 0, the smaller the difference between the two outlined contour sets.

[0079] Hausdorff distance (HD)

[0080] H(A,B)=max(h(A,B),h(B,A));

[0081] Among them, the bidirectional Hausdorff distance H(A, B) is the larger of the one-way distances h(A, B) and h(B, A), which measures the maximum mismatch between two point sets.

[0082] S4. Evaluation and optimization of model performance: Evaluate the model’s accuracy and other performance indicators, and optimize the model.

[0083] Specifically, the anatomical structure prediction model during radiotherapy is encapsulated and integrated into the software, and a software interface is developed to enable rapid input of multimodal images, and to call the encapsulated algorithm model to process the input images, and the results obtained after the processing are displayed in the software interface. Clinicians can conveniently use the software system to view the CBCT images and structures of each subsequent fraction during the patient's radiotherapy, and start the adaptive radiotherapy plan in advance, establish an adaptive radiotherapy trigger mechanism, and formulate a personalized treatment plan. Iterative software development and maintenance are carried out based on the results of clinical feedback to ensure that the software can be seamlessly integrated into the existing treatment process of the radiotherapy department and to ensure the clinical availability of the software output results.

[0084] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting changes in anatomical structures in a radiotherapy patient between radiotherapy fractions, characterized in that: The following steps are involved: Collection and management of imaging data: Collect the patient's imaging data before and after radiotherapy and establish a standardized data set; High-throughput imaging features extraction and analysis: Based on medical knowledge, imaging features related to radiotherapy response are selected, including morphology, texture, and signal intensity distribution features; Build a deep learning network model based on multimodal data to predict anatomical changes during radiotherapy, design and train a deep learning model to analyze imaging features and predict anatomical changes; Evaluation and optimization of model performance: Evaluate performance indicators such as model accuracy and optimize the model.

2. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 1, characterized in that: The image data acquisition comprises the following steps: Diagnostic multimodal MR images, localization CT images, CBCT images at days 1-3 and the first time in weeks 2-6, and radiotherapy plans were collected; Preprocess the CBCT images and convert them into pseudo-CT; The deep learning network is trained to predict the CBCT image at a certain time during the radiotherapy course, representing its anatomical image, and output the contour results of the target area and organs at risk on the CBCT.

3. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 2, characterized in that: The high-throughput imaging genomics feature extraction and analysis comprises the following steps: Perform high-throughput imaging genomics feature extraction and screening on multimodal MR images and positioning CT images; For the deep learning feature branch, multiple continuous stacks of 3D convolutional neural networks are used to screen key features; Multiple serial stacks and radiomics feature screening results were input into a multimodal hierarchical network.

4. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 3, characterized in that: The deep learning model training and optimization includes the following steps: Using pre-radiotherapy positioning CT, multimodal MR images, and the acquired CBCT images of each treatment fraction as input, the CBCT images and target volume and risk organ structures of each future fraction are predicted; Use deep learning-based networks to build multi-task models; CycelGAN is used to achieve end-to-end image prediction, and an improved segmentation model based on Unet is used to achieve contour prediction of tumors and endangered organs. The Encoder of the synthetic CBCT in cycelGAN shares a structure with the Encoder of the segmentation network. The Lossseg of the segmentation network is added to the LosscycelGAN of the cycelGAN network to form the Loss. The generation network and the segmentation network are trained together.

5. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 4, characterized in that: The quantitative evaluation indicators of anatomical images include MAE, RMSE, SSIM, and PSNR.

6. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 5, characterized in that: The quantitative evaluation indicators of anatomical structure include Dice and 95HD.

7. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 6, characterized in that: The anatomical structure change prediction includes three steps: feature extraction, multimodal feature aggregation and feature re-measurement, wherein: The feature extraction performs high-throughput imaging genomics feature extraction on multimodal MR images and positioning CT images. For deep learning feature branches, multiple continuous stacks of 3D convolutional neural networks are used to screen key features using feature dimensionality reduction methods such as PCA and Lasso, and the key features are input into the multimodal hierarchical network.

8. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 7, characterized in that: The multimodal feature aggregation uses two independent processing branches to perform feature learning; Among them, the stacked convolutional layers and downsampling layers perform feature learning layer by layer, and in this process, the features are kept in three-dimensional form; For the imaging omics feature processing branch, a one-dimensional fully connected neural network is used to perform feature learning layer by layer. During this process, the features are kept in one-dimensional form.

9. The method for predicting changes in in vivo anatomical structures of a radiotherapy patient between radiotherapy fractions according to claim 8, characterized in that: The feature weighting method utilizes the aggregated multimodal features to learn the weights when the features are fused, indicating the importance of the features of the two different modalities to the final anatomical image, thereby weighting the features of the two different modalities.

Images