A deep learning-based medulloblastoma multi-modal multi-view segmentation method

Abstract

The present application relates to the field of computer vision, and more particularly to a medulloblastoma multi-modal multi-view segmentation method based on deep learning, comprising the following steps: performing deep dimension splicing and fusion on a multi-modal slice library of axial, sagittal and coronal views, performing downsampling feature extraction and fusion operations, and then performing upsampling and decoding operations to obtain tumor three-dimensional labels of the three views, performing twice fusion and cropping on the tumor three-dimensional labels of the three views, and repeating the downsampling feature extraction, fusion, upsampling and decoding operations to generate a final three-dimensional tumor label. The present application uses multi-modal and multi-view magnetic resonance images, and through a two-stage segmentation method, rough segmentation is first performed, and then accurate segmentation is performed in combination with the original image, so as to achieve the segmentation capability of a professional doctor.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to a deep learning-based multimodal and multi-view segmentation method for medulloblastoma. Background Technology

[0002] Medulloblastoma is a common malignant tumor of the central nervous system, typically occurring in children and adolescents. Imaging can provide important information about medulloblastoma. On MRI (magnetic resonance imaging), the imaging morphology of medulloblastoma typically presents the following characteristics: Location: Medulloblastoma is most commonly found in the posteroinferior region of the cerebellum, known as the cerebellar vermis. It can also occur in areas such as the cerebellar hemisphere or brainstem. Enhancement pattern: Medulloblastoma usually shows significant enhancement after contrast administration. After contrast agent injection, the tumor usually shows homogeneous enhancement, but heterogeneous enhancement areas can also appear. Cystic changes: Cystic changes, i.e., the formation of cysts within the tumor, are common in medulloblastoma. These cysts may contain fluid and show low signal intensity. Intensity: On T1-weighted images, medulloblastoma usually shows isointense or slightly hyperintense signal intensity. On T2-weighted images, the tumor area usually shows hyperintense signal intensity. Boundaries: Medulloblastomas typically have irregular boundaries and may invade surrounding tissues.

[0003] With the development of artificial intelligence and the advancement of the biomedical field, research on medulloblastoma has entered a new era. Computers can now process large amounts of images, learn distributed features from them, and then map these features to a sample label space. Existing medulloblastoma segmentation methods are generally based on axial, single-view, single-modality segmentation.

[0004] Using a single-view, single-modal approach for medulloblastoma image segmentation has several drawbacks, including: Viewpoint limitation: A single viewpoint means viewing the image from only one fixed angle, which may result in some medulloblastoma regions being obscured or not clearly displayed. Certain important spatial relationships and morphological features may not be fully captured in a single viewpoint. Modal limitation: Single-modal segmentation utilizes only one image modality (such as MRI or CT), which may not fully utilize the information provided by other modalities. Different image modalities may offer different contrasts, resolutions, and tissue features; combining multimodal information can improve segmentation accuracy. Local information: Single-view, single-modal methods typically focus only on local information in the image, neglecting the importance of global context. The morphology, location, and relationship to surrounding tissues of medulloblastoma may require more global information for accurate segmentation. Image heterogeneity: Because medulloblastomas can have different morphologies and appearances in different patients and on different scanning devices, single-view, single-modal methods may perform poorly when dealing with image heterogeneity. This may lead to poor generalization ability across different datasets or in real-world applications. Segmentation accuracy and error: The segmentation accuracy of single-view, single-modal methods may be limited by the limitations of feature extraction and model design. In complex cases, segmentation errors, omissions, or missegments may occur. Summary of the Invention

[0005] This invention provides a deep learning-based multimodal and multi-view segmentation method for medulloblastoma. The aim is to construct a deep learning model to make accurate segmentations based on multimodal and multi-view MRI images of medulloblastoma, thereby helping doctors to develop more comprehensive treatment plans.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A deep learning-based multimodal, multi-view segmentation method for medulloblastoma includes the following steps:

[0008] S1, the T1 slice library, T1_E slice library, T2 slice library and T2_Flair slice library of axial view, sagittal view and coronal view respectively are spliced ​​in depth dimension to achieve multimodal fusion and obtain axial splicing fusion feature map, sagittal splicing fusion feature map and coronal splicing fusion feature map;

[0009] S2, perform downsampling feature extraction operations on the axial, sagittal, and coronal splicing and fusion feature maps respectively to obtain axial downsampling feature maps, sagittal downsampling feature maps, and coronal downsampling feature maps;

[0010] S3, the axial downsampled feature map, sagittal downsampled feature map and coronal downsampled feature map are respectively stitched and fused with two other view feature maps of the same size obtained in S2 to obtain the axial full view feature map, sagittal full view feature map and coronal full view feature map;

[0011] Upsampling feature extraction was performed on the axial, sagittal, and coronal full-view feature maps respectively to obtain axial upsampling feature maps, sagittal upsampling feature maps, and coronal upsampling feature maps;

[0012] S4, decode the axial upsampled feature map, sagittal upsampled feature map and coronal upsampled feature map respectively to obtain the axial tumor 3D label, sagittal tumor 3D label and coronal tumor 3D label;

[0013] The axial, sagittal, and coronal 3D tumor labels were fused to obtain a fused tumor 3D label.

[0014] S5, fuse the fused tumor 3D label with the axial, sagittal and coronal splicing fusion feature maps respectively to obtain the axial secondary fusion feature map, sagittal secondary fusion feature map and coronal secondary fusion feature map;

[0015] Brain contour regions were cropped from the axial, sagittal, and coronal secondary fusion feature maps, respectively, to obtain axial, sagittal, and coronal cropping feature maps.

[0016] S6, repeat step S2 for the axial, sagittal, and coronal cropping feature maps respectively; repeat step S3; repeat step S4; generate axial, sagittal, and coronal second-stage tumor 3D labels; fuse the axial, sagittal, and coronal second-stage tumor 3D labels to generate a final 3D tumor label.

[0017] Further, in step S1, the axial splicing and fusion feature map is represented as F i ax The sagittal splicing and fusion feature map is represented as F. i sag The coronal splicing and fusion feature map is represented as follows: The T1 slice library, after depth concatenation, is represented as follows: The T1_E slice library, after depth concatenation, is represented as: The T2 slice library, after depth-stitching, is represented as follows: The depth-stitched representation of the T2_Flair slice library is F. i t2f The specific description of the multimodal fusion is as follows:

[0018]

[0019] Where F i These are the fused feature maps, respectively, F i ax F i sag or Substituting into the calculation, Concat(·) represents a depth concatenation operation, where i represents the number of layers in the neural network.

[0020] Furthermore, in step S2, the axial splicing and fusion feature map F is processed respectively. i ax Sagittal splicing and fusion feature map Coronal splicing and fusion feature map The downsampling feature extraction operation is described in detail as follows:

[0021]

[0022]

[0023]

[0024] in It is an axial position downsampled feature map. It is a sagittal downsampling feature map. This is a coronal downsampling feature map, where Down(·) indicates the downsampling operation.

[0025] Furthermore, in step S3, the axial full-view feature map is as follows: Coronal full-view feature map Sagittal full-view feature map The specific description of the fusion operation is as follows:

[0026]

[0027]

[0028]

[0029] Axial upsampling feature map Sagittal upsampling feature map Coronal upsampling feature map Upsampling is specifically described as follows:

[0030]

[0031]

[0032]

[0033] Up(·) represents the upsampling operation.

[0034] Further, in step S4, the feature map of the axial position is sampled. Sagittal upsampling feature map Coronal upsampling feature map Decoding operations are performed separately, as described in detail below:

[0035]

[0036]

[0037]

[0038] Where pre ax It is a 3D label for axial tumors, pre cor It is a three-dimensional label for coronal tumors, pre sag These are the three-dimensional labels for the sagittal tumor, and Softmax(·) represents the Softmax activation function.

[0039] The prediction results from the three branches are fused. `pre` represents the fused tumor 3D labels, specifically described as follows:

[0040] pre = Add(pre) ax ,pre cor ,pre sag );

[0041] Add(·) represents the feature map addition and fusion operation.

[0042] Furthermore, in step S5, the fused tumor three-dimensional label pre is spliced ​​and fused with the axial feature map respectively. Sagittal splicing and fusion feature map F i sag Coronal splicing and fusion feature map The fusion process is specifically described as follows:

[0043]

[0044]

[0045]

[0046] in It is an axial downsampled fusion feature map. It is a coronal downsampling fusion feature map. It is a sagittal downsampling fusion feature map;

[0047] Subsample and fuse feature maps of axial positions respectively Sagittal downsampling fusion feature map Feature map fused with coronal downsampling The cropping process is described in detail as follows:

[0048]

[0049]

[0050]

[0051] in It is an axial position clipping feature map. This is a coronal clipping feature map. It is a sagittal clipping feature map, and Crop(·) represents the clipping operation.

[0052] Further, in step S6, a final three-dimensional tumor label is generated, specifically described as follows:

[0053] pre f =Add(pre ax_2 ,pre cor_2 ,pre sag_2 );

[0054] Where per ax_2 It is a second-stage axial three-dimensional label for tumors, per cor_2 It is a coronal phase II three-dimensional tumor label, per sag_2 It is a sagittal second-stage three-dimensional label of the tumor, per f It is the ultimate three-dimensional tumor marker.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1) Multiple Viewpoints: Using multi-view MRI imaging allows for observation of images from different angles, thus capturing more comprehensive morphological and locational information of medulloblastoma. This helps improve the accuracy and completeness of segmentation results.

[0057] 2) Modal fusion: Multimodal MRI images can fully utilize the complementary information between different modalities. Different modalities differ in contrast, resolution, and tissue features. By fusing multiple modalities, the robustness and accuracy of segmentation can be improved.

[0058] 3) Two-stage segmentation: This approach employs two stages—coarse segmentation and precise segmentation—which effectively reduces the complexity and computational cost of segmentation. The coarse segmentation stage quickly and initially locates the medulloblastoma region, while the precise segmentation stage utilizes the original image for more refined segmentation, combining the segmentation skills of professional physicians. This two-stage strategy improves segmentation accuracy while reducing computational resource requirements.

[0059] 4) Improved efficiency: By leveraging computer processing of large amounts of images and machine learning techniques, this method can automate medulloblastoma segmentation, reducing reliance on specialist physicians. This not only improves segmentation efficiency but also reduces medical costs and the consumption of human resources. Attached Figure Description

[0060] Figure 1 This is a schematic diagram illustrating the stitching results of different modalities in the depth channel according to an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the first-stage prediction results of the model provided in an embodiment of the present invention;

[0062] Figure 3 This is a diagram illustrating the overall framework of the model provided in this embodiment of the invention. Detailed Implementation

[0063] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Before describing the technical solutions of each embodiment of the present invention in detail, the terms and terms involved will be explained. In this specification, components with the same name or the same reference numerals represent similar or the same structures and are limited to illustrative purposes.

[0064] Terminology Explanation

[0065] 1) T1 (T1-weighted image): The T1 sequence is one of the most fundamental sequences in MRI. On a T1-weighted image, cerebrospinal fluid (CSF) appears black, adipose tissue appears white, and the contrast between gray and white matter is good. The T1 sequence is very useful for displaying anatomical structures and morphological features of brain tissue.

[0066] 2) T1-E (Enhanced T1-Weighted Imaging): The T1-E sequence is also known as an enhanced T1 sequence. In this sequence, the patient is injected beforehand with a substance called a contrast agent (usually a contrast agent). The contrast agent helps to highlight abnormal areas such as blood vessels, lesions, and inflammation, making these areas more clearly visible on the image.

[0067] 3) T2 (T2-weighted image): The T2 sequence is another commonly used type of MRI sequence. On a T2-weighted image, cerebrospinal fluid (CSF) appears white, gray matter appears medium gray, and white matter appears dark gray. The T2 sequence is very sensitive for displaying structures with high water content (such as cerebrospinal fluid, edema in tumors, etc.).

[0068] 4) T2-Flair (Liquid Attenuation Inversion Recovery Sequence): The T2-Flair sequence is an improved version of the T2 sequence. By using liquid saturation pulse technology, the influence of cerebrospinal fluid (CSF) signals can be suppressed. In the T2-Flair sequence, CSF appears black, while the contrast of other tissues is better, which is particularly useful for detecting lesions in brain tissue (such as cerebral infarction, abnormal CSF dynamics, etc.).

[0069] 5) MRI stands for Magnetic Resonance Imaging, a medical imaging technique used to generate high spatial resolution images of internal body structures. MRI uses strong magnetic fields and harmless radio waves to acquire images. It is based on the behavior of atomic nuclei (typically hydrogen nuclei in water molecules within the human body) in a magnetic field. When the human body is placed in a strong magnetic field, the atomic nuclei respond to the field, producing specific signals. By generating modulated magnetic fields around the patient's body and sending pulses of radio waves, MRI equipment can detect these signals from the atomic nuclei and convert them into detailed images. These images can reveal the structure, morphology, and function of internal organs, tissues, and lesions.

[0070] 6) concat is short for concatenate, which refers to the operation of joining two or more tensors along a specified axis. In deep learning, tensors are multidimensional arrays and can be regarded as an extension of matrices. When it is necessary to merge multiple tensors into a larger tensor, the concatenate operation or simply concat can be used.

[0071] 7) The UNet cascade model is a deep learning model for image segmentation tasks. Its structure is based on the UNet model but extended through cascading. UNet is a classic convolutional neural network (CNN) architecture, originally used for biomedical image segmentation. It features a symmetrical U-shaped structure containing a downsampling path (encoder) and an upsampling path (decoder). The encoder progressively extracts image features and reduces the resolution, while the decoder adjusts the feature maps back to the original resolution and generates the segmentation result. The UNet cascade model improves upon UNet by introducing multiple UNet modules for cascading. In the cascade model, each UNet module is chained together, with the output of one module serving as the input to the next. This design allows the model to progressively refine the segmentation results, from coarse to fine, to better capture details in the image. By cascading multiple UNet modules, the UNet cascade model can better handle image segmentation tasks. Each module learns feature representations at different scales at different levels, while fusing contextual information from the previous module, thereby improving the accuracy and detail preservation of the segmentation results. This cascaded structure helps address the balance between spatial local and global information in segmentation tasks. The UNet cascade model has achieved excellent performance in many image segmentation tasks, especially when precise segmentation of boundaries and fine structures is required. It has been widely used in medical image segmentation, such as tumor segmentation and organ segmentation. By cascading multiple UNet modules, the model can progressively improve its understanding of image details and contextual information, thereby obtaining more accurate segmentation results.

[0072] 8) PyTorch is an open-source machine learning framework used for building and training deep learning models. It was developed by Facebook's AI research team and published in 2016.

[0073] 9) In PyTorch, the input shape [N, C, D, H, W] of a 3D convolution represents a tensor with five dimensions. N: Batch Size, i.e., the number of input samples. During training, a batch of samples is typically input into the network for processing and parameter updates. C: Channels, i.e., the number of input feature maps for each sample. For grayscale images, the number of channels is 1. D: Depth, i.e., the dimension of the feature map in the depth direction. In 3D convolution, the feature map has not only height and width but also depth, used for processing volumetric or time-series data. H: Height, i.e., the vertical dimension of the feature map. W: Width, i.e., the horizontal dimension of the feature map.

[0074] Please see the appendix Figure 1-3 This invention discloses a deep learning-based multimodal, multi-view segmentation method for medulloblastoma, specifically including:

[0075] S1: As Figure 2 As shown, the model has three input branches: the first branch inputs the axial viewpoint, the second branch inputs the sagittal viewpoint mode, and the third branch inputs the coronal viewpoint mode. Then, the four modes T1, T1_E, T2, and T2_Flair of each input are concatenated and fused along the depth dimension.

[0076] S2: The three branch paths perform downsampling feature extraction operations respectively, generating feature maps from three perspectives after downsampling.

[0077] S3: Perform a concat feature fusion operation on the three viewpoint feature maps and all viewpoint feature maps of the same size as those in the downsampling process, and then perform an upsampling operation.

[0078] S4: As Figure 2 As shown, the three perspectives predict the three-dimensional tumor labels respectively after upsampling decoding, and the prediction results of the three perspectives are fused.

[0079] S5: The prediction results from S4 are fused with the feature maps generated by S1 to improve the expressive power of tumor region features. The fused features are then cropped to preserve the extent of the brain region, reducing the computational load of the model and improving the segmentation accuracy.

[0080] S6: Repeat steps S2, S3 and S4 on the output of S5 to generate an accurate three-dimensional tumor label from the prediction results.

[0081] S7: The model is trained using medulloblastoma MRI training data to generate a precise three-dimensional tumor label. The MRI training data consists of preoperative medulloblastoma MRI images and the tumor's three-dimensional label. Further, in step S1, the multimodal fusion method is depth stitching, and the axial features are represented as... The characteristic of the sagittal position is represented as Coronal features are represented as The T1 modal characteristics are represented as follows: The T1_E modal features are represented as follows: The T2 modal characteristics are represented as follows: The T2_Flair modal features are represented as follows: Multimodal fusion is specifically described as follows:

[0082]

[0083] Where F i It is the fused feature map, where i represents the number of layers in the neural network.

[0084] Further in step S2, the features after multimodal fusion are downsampled, specifically as follows:

[0085]

[0086]

[0087]

[0088] in It is a feature map of the axial position. It is a feature map of the coronal plane, F i sag It is a feature map of the sagittal plane.

[0089] Further in step S3, the three-view information is fused. The fusion method involves stitching the feature maps from different views together in the depth channel. The axially fused feature is... The characteristics of coronal fusion are The characteristics of sagittal fusion are Specifically described as follows:

[0090]

[0091]

[0092]

[0093] Furthermore, in step S4, the fused three-view images are first upsampled, specifically as follows:

[0094]

[0095]

[0096]

[0097] The upsampled information is then decoded to predict the tumor label, specifically described as follows:

[0098]

[0099]

[0100]

[0101] Where per ax It is a tumor label predicted in the axial view, pre cor It is the tumor label predicted in the coronal region, presag It is a tumor label predicted in the sagittal plane.

[0102] Finally, the prediction results of the three branches are fused. `pre` represents the fused label result, specifically described as follows:

[0103] per = Add(per ax ,pre cor ,pre sag ).

[0104] Furthermore, in step S5, the prediction results of the first-stage network and the features of S1 are fused, specifically as follows:

[0105]

[0106]

[0107]

[0108] in It is a fusion of the prediction results and the first-stage axial features. It is a fusion of the prediction results and the coronal features of the first stage. It is a fusion of the prediction results and the first-stage sagittal features.

[0109] The merged data is then trimmed to reduce model parameters and computation time. Specifically:

[0110]

[0111]

[0112]

[0113] in It is a feature map of axial position reduction. This is a characteristic image of coronal clipping. It is a feature map of sagittal clipping.

[0114] Furthermore, in step S6, the processes S2 to S4 of the first stage are repeated, ultimately generating a precise three-dimensional tumor label, specifically described as follows:

[0115] pre f =Add(pre ax_2 ,pre cor_2 ,pre sag_2 );

[0116] Where pre ax_2 It is the tumor label predicted in the second stage of axial imaging, pre cor_2It is the tumor label predicted in the second stage of coronal imaging, pre sag_2 It is the tumor label predicted in the second stage of sagittal imaging, pre f It is the final tumor prediction label.

[0117] Concat(·) represents depth concatenation, Add(·) represents feature map addition and fusion, Up(·) represents upsampling, Down(·) represents downsampling, Softmax(·) represents the Softmax activation function, and Crop(·) represents cropping.

[0118] The specific technical solution adopted in this invention patent is as follows:

[0119] 1. Data Preprocessing

[0120] Each patient's data includes four modalities: T1, T1_E, T2, and T2_Flair. It also includes three perspectives: axial, sagittal, and coronal.

[0121] Because multimodal and multi-view data from the same patient are used, it is necessary to address issues such as misalignment of study cohort size, image dimensions, resolution, and voxel intensity slices for various modalities within the same patient. This invention employs an image registration method, selecting the T1 modality of each patient's transverse view as a baseline, and registering the modalities of other viewpoints of the same patient onto the T1 modality of the transverse view. The registered image size, dimensions, resolution, and voxel intensity will be consistent with the T1 modality, and different data slices will also be aligned.

[0122] 2. The process of dissecting medulloblastoma:

[0123] Images of the same patient from different modalities at the same viewpoint are stitched together using the depth channel. For example, in PyTorch, the 3D convolution input is [N,C,D,H,W], and in this method, C is 4, representing four modalities. Data from different viewpoints are output in three separate paths, and finally, data fusion is performed at the prediction stage.

[0124] The model consists of a two-stage cascaded UNet model. The first stage processes multimodal, multi-view data, outputting a coarse segmentation result. This segmentation result is then fused with the initial downsampled data to enhance the tumor region's features. This fusion is then fed into the second stage of the model, generating precise 3D segmentation labels for the tumor.

[0125] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A deep learning-based multimodal, multi-view segmentation method for medulloblastoma, characterized in that, Includes the following steps: S1, the T1 slice library, T1_E slice library, T2 slice library and T2_Flair slice library of axial view, sagittal view and coronal view respectively are spliced ​​in depth dimension to achieve multimodal fusion and obtain axial splicing fusion feature map, sagittal splicing fusion feature map and coronal splicing fusion feature map; S2, perform downsampling feature extraction operations on the axial, sagittal, and coronal splicing and fusion feature maps respectively to obtain axial downsampling feature maps, sagittal downsampling feature maps, and coronal downsampling feature maps; S3, the axial downsampled feature map, sagittal downsampled feature map and coronal downsampled feature map are respectively stitched and fused with two other view feature maps of the same size obtained in S2 to obtain the axial full view feature map, sagittal full view feature map and coronal full view feature map; Upsampling feature extraction was performed on the axial, sagittal, and coronal full-view feature maps respectively to obtain axial upsampling feature maps, sagittal upsampling feature maps, and coronal upsampling feature maps; S4, decode the axial upsampled feature map, sagittal upsampled feature map and coronal upsampled feature map respectively to obtain the axial tumor 3D label, sagittal tumor 3D label and coronal tumor 3D label; The axial, sagittal, and coronal 3D tumor labels were fused to obtain a fused tumor 3D label. S5, fuse the fused tumor 3D label with the axial, sagittal and coronal splicing fusion feature maps respectively to obtain the axial secondary fusion feature map, sagittal secondary fusion feature map and coronal secondary fusion feature map; Brain contour regions were cropped from the axial, sagittal, and coronal secondary fusion feature maps, respectively, to obtain axial, sagittal, and coronal cropping feature maps. S6, repeat step S2 for the axial, sagittal, and coronal cropping feature maps respectively; repeat step S3; repeat step S4; generate axial, sagittal, and coronal second-stage tumor 3D labels; fuse the axial, sagittal, and coronal second-stage tumor 3D labels to generate a final 3D tumor label.

2. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 1, characterized in that, In step S1, the axial splicing and fusion feature map is represented as follows: The sagittal splicing and fusion feature map is represented as The coronal splicing and fusion feature map is represented as follows: The T1 slice library depth is represented as The T1_E slice library, after depth concatenation, is represented as The T2 slice library, after depth stitching, is represented as The T2_Flair slice library, after depth concatenation, is represented as The specific description of the multimodal fusion is as follows: ； in These are the fused feature maps, which are respectively... , or Substitute into the calculation, This represents a depthwise concatenation operation, where i represents the number of layers in the neural network.

3. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 2, characterized in that, In step S2, the axial position splicing and fusion feature maps are respectively... Sagittal splicing and fusion feature map Coronal splicing and fusion feature map The downsampling feature extraction operation is described in detail as follows: ； ； ； in It is an axial position downsampled feature map. It is a sagittal downsampling feature map. It is a coronal downsampling feature map. This indicates a downsampling operation.

4. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 3, characterized in that, In step S3, the axial full-view feature map is as follows: The coronal full-view feature map is The sagittal full-view feature map is The specific description of the fusion operation is as follows: ； ； ； Axial upsampling feature map The sagittal upsampling feature map is The coronal upsampling feature map is Upsampling is specifically described as follows: ； ； ； in This indicates an upsampling operation.

5. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 4, characterized in that, In step S4, the feature map of the axial position is sampled. Sagittal upsampling feature map Coronal upsampling feature map Decoding operations are performed separately, as described in detail below: ； ； ； in It is a 3D label for axial tumors. It is a three-dimensional label of a coronal tumor. It is a three-dimensional label for sagittal tumors. This represents the Softmax activation function; The prediction results from the three branches are then fused. It is a fusion of tumor 3D labels, specifically described as follows: ； in This indicates a feature map addition and fusion operation.

6. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 5, characterized in that, In step S5, the fused tumor three-dimensional tags are... Feature maps spliced ​​and fused with axial positions respectively Sagittal splicing and fusion feature map Coronal splicing and fusion feature map The fusion process is specifically described as follows: ； ； ； in It is an axial downsampled fusion feature map. It is a coronal downsampling fusion feature map. It is a sagittal downsampling fusion feature map; Subsample and fuse feature maps of axial positions respectively Sagittal downsampling fusion feature map Feature map fused with coronal downsampling The cropping process is described in detail as follows: ； ； ； in It is an axial position clipping feature map. This is a coronal clipping feature map. It is a sagittal clipping feature map. This indicates a cropping operation.

7. The deep learning-based multimodal, multi-view segmentation method for medulloblastoma according to claim 6, characterized in that, In step S6, a final three-dimensional tumor label is generated, specifically described as follows: ； in This is a second-stage axial three-dimensional label of the tumor. It is a coronal phase II three-dimensional tumor label. This is a sagittal second-stage three-dimensional label of the tumor. It is the ultimate three-dimensional tumor marker.

Images