Multi-modal medical image fusion system based on deep learning

Abstract

The invention discloses a multi-modal medical image fusion system based on deep learning, and relates to the technical field of medical images, and the system comprises an image collection module which obtains medical image data from different modalities; the data preprocessing module is used for carrying out denoising, standardization, image enhancement and registration on the acquired image data; the deep learning fusion module is used for carrying out feature extraction and information fusion on the image data of different modalities based on a convolutional neural network; the diagnosis reasoning module is used for evaluating potential disease information in the medical image data through a trained classification and regression model based on the fused image data; and the visualization module is used for visually displaying the processed image data and the diagnosis result and providing an image and a diagnosis report. According to the method, the limitation of a single mode is overcome by fusing the multi-mode images, more comprehensive and accurate diagnosis information is provided, feature extraction and fusion are performed by adopting the convolutional neural network, and the disease recognition precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a multimodal medical image fusion system based on deep learning. Background Art

[0002] With the rapid development of medical imaging technology, medical imaging modalities such as CT, MRI, ultrasound, and X-rays are widely used in clinical diagnosis. Each imaging modality has its own specific advantages and disadvantages. For example, CT images can provide detailed information on bone structure, while MRI images excel in soft tissue imaging. Ultrasound images are highly real-time, and X-ray images are mainly used to observe structures such as bones and chest. However, the imaging data of each modality has its own limitations, such as resolution, contrast, and noise. This makes it difficult for single-modality images to provide comprehensive disease diagnostic information.

[0003] In order to overcome these problems, multimodal medical image fusion technology based on deep learning has emerged. By fusing data from different imaging modalities, this technology can integrate the advantages of each modality, fully explore potential information, and improve the accuracy and reliability of disease diagnosis. Especially in complex medical imaging data, deep learning methods can help discover potential correlations in different modal data by automatically learning high-level features and information fusion, providing more comprehensive disease diagnosis support. Traditional medical image fusion methods usually rely on manually designed features and rules, which is not only time-consuming and inefficient, but also easily limited by manual experience. In contrast, deep learning methods based on convolutional neural networks can automatically learn complex features from data, thereby achieving more efficient and accurate multimodal image fusion. Therefore, multimodal medical image fusion systems based on deep learning have significant application potential in medical image analysis, auxiliary diagnosis, and clinical decision support. Summary of the Invention

[0004] In order to solve the above technical problems, a multimodal medical image fusion system based on deep learning is provided. This technical solution solves the above problems.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] Multimodal medical image fusion system based on deep learning, including:

[0007] Image acquisition module, which acquires medical imaging data from different modalities, including CT images, MRI images, ultrasound images, and X-ray images;

[0008] A data preprocessing module is electrically connected to the image acquisition module and is used to perform denoising, standardization, image enhancement, and registration on the acquired image data to achieve spatial alignment and feature consistency of image data of different modalities;

[0009] Deep learning fusion module, which is used to extract features and fuse information from image data of different modalities based on convolutional neural networks;

[0010] A diagnostic reasoning module, which is electrically connected to the deep learning fusion module. The diagnostic reasoning module is used to evaluate potential disease information in medical imaging data based on the fused imaging data using trained classification and regression models, and provide auxiliary diagnostic advice to doctors.

[0011] The visualization module is electrically connected to the diagnosis and reasoning module, and is used to visualize the processed image data and diagnosis results and provide images and diagnosis reports.

[0012] Preferably, the image acquisition module specifically includes:

[0013] Image acquisition unit: acquires data from different types of medical imaging equipment, including CT scanners, MRI equipment, ultrasound probes, and X-ray equipment;

[0014] Data transmission unit: transmits image data from the acquisition device to the system's processing platform and converts images of different modalities into a unified format;

[0015] Modality recognition unit: automatically identifies and labels the modality type of each acquired image data;

[0016] Synchronous control unit: synchronizes the acquisition time of image data of different modalities to avoid image registration errors caused by time asynchrony.

[0017] Preferably, the data preprocessing module specifically includes:

[0018] Denoising unit: performs noise filtering on the collected image data and uses filtering methods to remove noise points in the image;

[0019] Image standardization unit: converts image data of different modalities into a unified standard scale and resolution to keep the image data consistent in space;

[0020] Image registration unit: performs spatial registration on image data of different modalities so that the images of each modality are accurately aligned in the same coordinate system.

[0021] Preferably, the deep learning fusion module includes:

[0022] Feature extraction unit: uses convolutional neural network to perform deep feature extraction on each modality image data to obtain high-level feature identification of each image;

[0023] Fusion algorithm unit: Through adaptive weight mechanism or multi-layer fusion strategy, it fuses high-level features of different modalities, thereby comprehensively utilizing the complementary information in each modal data;

[0024] Information fusion output unit: outputs the final multimodal fusion result based on the fused feature data for use by subsequent diagnostic reasoning modules.

[0025] Preferably, the use of a convolutional neural network to perform deep feature extraction on each modality image data to obtain high-level feature labels for each image specifically includes:

[0026] The image data is passed into the network as input. The image data will pass through a series of convolutional layers to extract low-level features. The convolution kernel scans the input image to generate a feature map.

[0027] After the convolution layer, a pooling layer is added to perform maximum pooling, reduce the spatial resolution of the feature map, extract the most important information from the local area, and output the pooled feature map;

[0028] The convolutional layers and pooling layers are repeatedly stacked to gradually extract more and more abstract features, and gradually learn from local features to global features;

[0029] The convolutional neural network adds a fully connected layer at the end to flatten the multi-dimensional feature map and map it to a higher-level feature space to generate the output of the fully connected layer;

[0030] After being processed by the convolutional neural network, the image data of each modality is extracted through the corresponding convolutional neural network to obtain the corresponding high-level feature representation, and the high-level features of the image of each modality will be passed to the subsequent fusion module.

[0031] Preferably, the method of fusing high-level features of different modalities through an adaptive weight mechanism or a multi-layer fusion strategy, thereby comprehensively utilizing the complementary information in the data of each modality, specifically includes:

[0032] The original features of different modalities are spliced at the input stage to form a unified feature representation. At the final stage of the model, the features of each modality are fused and the final fused features are obtained by weighted averaging.

[0033] Introducing an adaptive weight model to automatically adjust weights according to different tasks and input data;

[0034] Based on the multi-layer fusion strategy, the features of different modalities are gradually fused through multiple fusion layers. For classification tasks, the final classification result is obtained through the fully connected layer and activation function.

[0035] Preferably, the introduction of the adaptive weight model to automatically adjust the weight according to different tasks and input data specifically includes:

[0036] Among them, the adaptation weight model formula is:

[0037] α i =σ(w i F m +b i )

[0038] Where w i and b i are the learned weights and biases, σ is the activation function, which represents the normalization of the weights to ensure that the sum of the weights is 1, and F m is the high-level feature of the i-th modality, α i is the adaptive weight of the i-th mode;

[0039] Among them, the expression of feature fusion is:

[0040]

[0041] Where n is the number of modes, α i is the adaptive weight of the i-th mode, F f is the feature representation after fusion.

[0042] Preferably, the diagnostic reasoning module specifically includes:

[0043] Disease recognition unit: uses a deep learning model to perform disease recognition on the fused image data, automatically identifying the lesion area and disease type in the image;

[0044] Risk Assessment Unit: Based on the disease identification results and combined with other clinical data of the patient, it quantitatively assesses the potential disease risk and generates a risk index for disease development;

[0045] The clinical advice unit provides doctors with specific clinical decision-making recommendations based on the diagnostic results and risk assessment output by the reasoning model, supporting the formulation of personalized treatment plans.

[0046] Preferably, the quantitative assessment of potential disease risk based on the disease identification results in combination with other clinical data of the patient to generate a risk index for disease development specifically includes:

[0047] The risk index calculation formula is:

[0048] R=β0+β1Y+β2C1+β3C2+…+β n C n

[0049] In the formula, R is the risk index, β0 is the constant term, Y is the disease identification result, C1 is the patient's age, C2 is the patient's gender, and C n is the nth clinical data feature of the patient, β1, β2, β n is the weight.

[0050] Preferably, the visualization module specifically includes:

[0051] Image display unit: Visually display the pre-processed and fused image data, including image comparison, zoom in and zoom out interactive functions;

[0052] Diagnosis result display unit: presents the disease information, risk assessment results and treatment recommendations generated by the diagnostic reasoning module in graphical and textual form;

[0053] Multimodal fusion display unit: Simultaneously displays medical images of different modalities and their fusion results, intuitively demonstrating the complementarity and fusion effect of each modality information.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention proposes that by integrating the advantages of different types of medical images, it can provide doctors with more accurate and comprehensive diagnostic support, greatly improving the efficiency of disease identification and clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a system framework diagram of the present invention;

[0057] Figure 2 This is the internal system framework diagram of the deep learning fusion module in the present invention. DETAILED DESCRIPTION

[0058] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0059] Reference Figure 1 As shown in the figure, the multimodal medical image fusion system based on deep learning includes:

[0060] Image acquisition module, which acquires medical imaging data from different modalities, including CT images, MRI images, ultrasound images, and X-ray images;

[0061] A data preprocessing module is electrically connected to the image acquisition module and is used to perform denoising, standardization, image enhancement, and registration on the acquired image data to achieve spatial alignment and feature consistency of image data of different modalities;

[0062] Deep learning fusion module, which is used to extract features and fuse information from image data of different modalities based on convolutional neural networks;

[0063] The diagnostic reasoning module is electrically connected to the deep learning fusion module. The diagnostic reasoning module is used to evaluate potential disease information in medical imaging data based on the fused image data and through trained classification and regression models, providing auxiliary diagnosis opinions for doctors.

[0064] The visualization module is electrically connected to the diagnosis and reasoning module. The visualization module is used to visualize the processed image data and diagnosis results and provide images and diagnosis reports.

[0065] The image acquisition module specifically includes:

[0066] Image acquisition unit: acquires data from different types of medical imaging equipment, including CT scanners, MRI equipment, ultrasound probes, and X-ray equipment;

[0067] Data transmission unit: transmits image data from the acquisition device to the system's processing platform and converts images of different modalities into a unified format;

[0068] Modality recognition unit: automatically identifies and labels the modality type of each acquired image data;

[0069] Synchronous control unit: synchronizes the acquisition time of image data of different modalities to avoid image registration errors caused by time asynchrony;

[0070] It combines various types of medical imaging equipment and has a unified format conversion function, supports seamless data connection between devices, and reduces the problem of inconsistent data formats caused by different devices.

[0071] The data preprocessing module specifically includes:

[0072] Denoising unit: performs noise filtering on the collected image data and uses filtering methods to remove noise points in the image;

[0073] Image standardization unit: converts image data of different modalities into a unified standard scale and resolution to keep the image data consistent in space;

[0074] Image registration unit: performs spatial registration on image data of different modalities so that the images of each modality are accurately aligned in the same coordinate system;

[0075] Data preprocessing not only includes denoising, normalization, and image enhancement, but also adds precise spatial registration steps to ensure the spatial consistency of images of different modalities, providing an accurate basis for subsequent feature fusion.

[0076] Reference Figure 2 As shown, the deep learning fusion module includes:

[0077] Feature extraction unit: uses convolutional neural network to perform deep feature extraction on each modality image data to obtain high-level feature identification of each image;

[0078] Fusion algorithm unit: Through adaptive weight mechanism or multi-layer fusion strategy, it fuses high-level features of different modalities, thereby comprehensively utilizing the complementary information in each modal data;

[0079] Information fusion output unit: outputs the final multimodal fusion result based on the fused feature data for use by subsequent diagnostic reasoning modules;

[0080] By automatically extracting image features through deep convolutional neural networks and adopting a multi-level fusion strategy, it is possible to deeply mine the complementary information of images of different modalities, greatly improving the effect of data fusion and the accuracy of diagnosis.

[0081] A convolutional neural network is used to extract deep features from the image data of each modality, and the high-level feature identification of each image is obtained, including:

[0082] The image data is passed into the network as input. The image data will pass through a series of convolutional layers to extract low-level features. The convolution kernel scans the input image to generate a feature map.

[0083] After the convolution layer, a pooling layer is added to perform maximum pooling, reduce the spatial resolution of the feature map, extract the most important information from the local area, and output the pooled feature map;

[0084] The convolutional layers and pooling layers are repeatedly stacked to gradually extract more and more abstract features, and gradually learn from local features to global features;

[0085] The convolutional neural network adds a fully connected layer at the end to flatten the multi-dimensional feature map and map it to a higher-level feature space to generate the output of the fully connected layer;

[0086] After being processed by the convolutional neural network, the image data of each modality is extracted through the corresponding convolutional neural network to obtain the corresponding high-level feature representation, and the high-level features of the image of each modality are passed to the subsequent fusion module;

[0087] The convolutional neural network is used for deep feature extraction, and combined with the pooling layer and the fully connected layer, it effectively reduces redundant features and improves computational efficiency and accuracy. Through high-level feature representation, the model can better identify potential disease characteristics.

[0088] Through adaptive weighting mechanisms or multi-layer fusion strategies, high-level features of different modalities are fused to comprehensively utilize the complementary information in each modality data. Specifically,

[0089] The original features of different modalities are spliced at the input stage to form a unified feature representation. At the final stage of the model, the features of each modality are fused and the final fused features are obtained by weighted averaging.

[0090] Introducing an adaptive weight model to automatically adjust weights according to different tasks and input data;

[0091] Based on the multi-layer fusion strategy, the features of different modalities are gradually integrated through multiple fusion layers. For classification tasks, the final classification result is obtained through the fully connected layer and activation function.

[0092] The weights are automatically adjusted according to the characteristics of different modalities and input data, which has strong adaptability and can optimize the model for different tasks and image data, thereby improving the model's fusion effect on information from different image modalities.

[0093] Introducing an adaptive weight model to automatically adjust weights based on different tasks and input data. Specifically:

[0094] Among them, the adaptation weight model formula is:

[0095] α i =σ(w i F m +b i )

[0096] Where w i and b i are the learned weights and biases, σ is the activation function, which represents the normalization of the weights to ensure that the sum of the weights is 1, and F m is the high-level feature of the i-th modality, α i is the adaptive weight of the i-th mode;

[0097] Among them, the expression of feature fusion is:

[0098]

[0099] Where n is the number of modes, α i is the adaptive weight of the i-th mode, F f is the feature representation after fusion.

[0100] The diagnostic reasoning module specifically includes:

[0101] Disease recognition unit: uses a deep learning model to perform disease recognition on the fused image data, automatically identifying the lesion area and disease type in the image;

[0102] Risk Assessment Unit: Based on the disease identification results and combined with other clinical data of the patient, it quantitatively assesses the potential disease risk and generates a risk index for disease development;

[0103] The clinical advice unit provides doctors with specific clinical decision-making recommendations based on the diagnostic results and risk assessment output by the inference model, supporting the formulation of personalized treatment plans;

[0104] The introduction of a quantitative risk assessment model, combined with the patient's age, gender, and clinical data, can provide doctors with more accurate disease prediction and risk assessment. At the same time, it automatically identifies the lesion area and reduces manual diagnostic errors.

[0105] Based on the disease identification results and combined with other clinical data of the patient, a quantitative assessment of potential disease risks is conducted to generate a risk index for disease development, specifically including:

[0106] The risk index calculation formula is:

[0107] R=β0+β1Y+β2C1+β3C2+…+β n C n

[0108] In the formula, R is the risk index, β0 is the constant term, Y is the disease identification result, C1 is the patient's age, C2 is the patient's gender, and C n is the nth clinical data feature of the patient, β1, β2, β n is the weight.

[0109] The visualization module specifically includes:

[0110] Image display unit: Visually display the pre-processed and fused image data, including image comparison, zoom in and zoom out interactive functions;

[0111] Diagnosis result display unit: presents the disease information, risk assessment results and treatment recommendations generated by the diagnostic reasoning module in graphical and textual form;

[0112] Multimodal fusion display unit: Simultaneously displays medical images of different modalities and their fusion results, intuitively demonstrating the complementarity and fusion effect of each modality information.

[0113] In summary, the advantages of the present invention are:

[0114] By fusing imaging data from different modalities, such as CT, MRI, ultrasound, and X-rays, the system can fully leverage the strengths of each modality, overcome the limitations of single-modality data, and provide more comprehensive diagnostic information. For example, CT and X-rays primarily provide information about bone structure, while MRI and ultrasound can better display soft tissue details. The fused images can provide doctors with more accurate and comprehensive disease information.

[0115] Using convolutional neural networks to extract and fuse features from image data can automatically learn high-level feature representations from large amounts of data, thereby achieving accurate disease identification. Deep learning models can identify complex information hidden in images of different modalities, avoiding the limitations of manual feature extraction and providing higher diagnostic accuracy.

[0116] Fused imaging data can improve the accuracy of disease identification, especially in the diagnosis of complex diseases. For example, in tumor detection, CT images can clearly show the tumor's morphology, while MRI images provide more information about the tissue structure. The combination of the two can help doctors better determine the nature, size, and location of the tumor.

[0117] The visualization module can display processed images and diagnostic results in the form of graphics and text, helping doctors to understand the analysis results more intuitively. Through interactive functions such as image comparison, zooming in and out, doctors can more conveniently view data and fusion effects of different modalities, further supporting personalized diagnosis and treatment plans.

[0118] The diagnostic reasoning module not only automatically identifies lesions in images but also performs risk assessment based on disease identification results and the patient's clinical data, providing doctors with a quantitative risk index. This function helps doctors assess the potential risk of disease progression, make timely clinical decisions, and develop personalized treatment plans.

[0119] The entire system automates the processing of image data, from data acquisition, preprocessing, feature extraction, information fusion to diagnostic reasoning, significantly improving work efficiency and reducing the time and cost of manual operations. Furthermore, the adaptive adjustment capabilities of deep learning models enable the system to flexibly optimize model performance based on different tasks and input data, further enhancing diagnostic accuracy.

[0120] Through deep learning fusion of multimodal medical images, the system can detect subtle lesions in images at an early stage and improve the ability to detect diseases early. In the early stages of certain diseases, traditional imaging technology may find it difficult to identify lesions. By fusing imaging data from different modalities, the system can provide more sensitive and accurate detection results, providing doctors with effective early warning information and promoting early diagnosis and treatment of diseases.

[0121] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the invention as claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal medical image fusion system based on deep learning, characterized by: include: Image acquisition module, which acquires medical imaging data from different modalities, including CT images, MRI images, ultrasound images, and X-ray images; A data preprocessing module is electrically connected to the image acquisition module and is used to perform denoising, standardization, image enhancement, and registration on the acquired image data to achieve spatial alignment and feature consistency of image data of different modalities; Deep learning fusion module, which is used to extract features and fuse information from image data of different modalities based on convolutional neural networks; A diagnostic reasoning module, which is electrically connected to the deep learning fusion module. The diagnostic reasoning module is used to evaluate potential disease information in medical imaging data based on the fused imaging data using trained classification and regression models, and provide auxiliary diagnostic advice to doctors. The visualization module is electrically connected to the diagnosis and reasoning module, and is used to visualize the processed image data and diagnosis results and provide images and diagnosis reports.

2. The multimodal medical image fusion system based on deep learning according to claim 1, characterized in that: The image acquisition module specifically includes: Image acquisition unit: acquires data from different types of medical imaging equipment, including CT scanners, MRI equipment, ultrasound probes, and X-ray equipment; Data transmission unit: transmits image data from the acquisition device to the system's processing platform and converts images of different modalities into a unified format; Modality recognition unit: automatically identifies and labels the modality type of each acquired image data; Synchronous control unit: synchronizes the acquisition time of image data of different modalities to avoid image registration errors caused by time asynchrony.

3. The multimodal medical image fusion system based on deep learning according to claim 2, characterized in that: The data preprocessing module specifically includes: Denoising unit: performs noise filtering on the collected image data and uses filtering methods to remove noise points in the image; Image standardization unit: converts image data of different modalities into a unified standard scale and resolution to keep the image data consistent in space; Image registration unit: performs spatial registration on image data of different modalities so that the images of each modality are accurately aligned in the same coordinate system.

4. The multimodal medical image fusion system based on deep learning according to claim 3, characterized in that: The deep learning fusion module includes: Feature extraction unit: uses convolutional neural network to perform deep feature extraction on each modality image data to obtain high-level feature identification of each image; Fusion algorithm unit: Through adaptive weight mechanism or multi-layer fusion strategy, it fuses high-level features of different modalities, thereby comprehensively utilizing the complementary information in each modal data; Information fusion output unit: outputs the final multimodal fusion result based on the fused feature data for use by subsequent diagnostic reasoning modules.

5. The multimodal medical image fusion system based on deep learning according to claim 4, characterized in that: The use of convolutional neural networks to extract deep features from image data of each modality to obtain high-level feature representations of each image specifically includes: The image data is passed into the network as input. The image data will pass through a series of convolutional layers to extract low-level features. The convolution kernel scans the input image to generate a feature map. After the convolution layer, a pooling layer is added to perform maximum pooling, reduce the spatial resolution of the feature map, extract the most important information from the local area, and output the pooled feature map; The convolutional layers and pooling layers are repeatedly stacked to gradually extract more and more abstract features, and gradually learn from local features to global features; The convolutional neural network adds a fully connected layer at the end to flatten the multi-dimensional feature map and map it to a higher-level feature space to generate the output of the fully connected layer; After being processed by the convolutional neural network, the image data of each modality is extracted through the corresponding convolutional neural network to obtain the corresponding high-level feature representation, and the high-level features of the image of each modality will be passed to the subsequent fusion module.

6. The multimodal medical image fusion system based on deep learning according to claim 5, characterized in that: The method of fusing high-level features of different modalities through an adaptive weight mechanism or a multi-layer fusion strategy, thereby comprehensively utilizing the complementary information in the data of each modality, specifically includes: The original features of different modalities are spliced at the input stage to form a unified feature representation. At the final stage of the model, the features of each modality are fused and the final fused features are obtained by weighted averaging. Introducing an adaptive weight model to automatically adjust weights according to different tasks and input data; Based on the multi-layer fusion strategy, the features of different modalities are gradually fused through multiple fusion layers. For classification tasks, the final classification result is obtained through the fully connected layer and activation function.

7. The multimodal medical image fusion system based on deep learning according to claim 6, characterized in that: The adaptive weight model is introduced to automatically adjust the weight according to different tasks and input data. include: Among them, the adaptation weight model formula is: a i =σ(w i F m +b i ) Where w i and b i are the learned weights and biases, σ is the activation function, which represents the normalization of the weights to ensure that the sum of the weights is 1, and F m is the high-level feature of the i-th modality, α i is the adaptive weight of the i-th mode; Among them, the expression of feature fusion is: Where n is the number of modes, α i is the adaptive weight of the i-th mode, F f is the feature representation after fusion.

8. The multimodal medical image fusion system based on deep learning according to claim 7, characterized in that: The diagnostic reasoning module specifically includes: Disease recognition unit: uses a deep learning model to perform disease recognition on the fused image data, automatically identifying the lesion area and disease type in the image; Risk Assessment Unit: Based on the disease identification results and combined with other clinical data of the patient, it quantitatively assesses the potential disease risk and generates a risk index for disease development; The clinical advice unit provides doctors with specific clinical decision-making recommendations based on the diagnostic results and risk assessment output by the reasoning model, supporting the formulation of personalized treatment plans.

9. The multimodal medical image fusion system based on deep learning according to claim 8, characterized in that: Based on the disease identification results, combined with other clinical data of the patient, a quantitative assessment of the potential disease risk is performed to generate a specific risk index for disease development. include: The risk index calculation formula is: R=β0+β1Y+β2C1+β3C2+…+β n C n In the formula, R is the risk index, β0 is the constant term, Y is the disease identification result, C1 is the patient's age, C2 is the patient's gender, and C n is the nth clinical data feature of the patient, β1, β2, β n is the weight.

10. The multimodal medical image fusion system based on deep learning according to claim 9, characterized in that: The visualization module specifically includes: Image display unit: Visually display the pre-processed and fused image data, including image comparison, zoom in and zoom out interactive functions; Diagnosis result display unit: presents the disease information, risk assessment results and treatment recommendations generated by the diagnostic reasoning module in graphical and textual form; Multimodal fusion display unit: Simultaneously displays medical images of different modalities and their fusion results, intuitively demonstrating the complementarity and fusion effect of each modality information.

Images