A deep fusion learning-assisted demyelination analysis method for children's brain MRI

Abstract

The present invention discloses a method for analyzing demyelination in children's brain MRI by deep fusion learning. First, preprocessing of children's demyelinating disease data and data set construction are performed; then the auxiliary model design of deep fusion learning demyelinating disease is performed; the auxiliary model of deep fusion learning demyelinating disease is trained by the constructed children's demyelinating disease data set; finally, the acquisition and feature classification of high-signal white matter segmentation map are completed by the trained deep fusion learning demyelinating disease auxiliary model, and the high-signal white matter segmentation map and feature classification labels are displayed by the visualization result model. The method of the present invention can not only assist doctors in clinical diagnosis, but also greatly improve the rate of image annotation, which will greatly shorten the diagnosis time and diagnosis cost of patients, enable patients to obtain timely treatment, alleviate the neurological deficits left over from the patient's prognosis process, and obtain better prognosis effects.

Description

Technical Field

[0001] The present invention relates to deep learning, computer vision, medical image segmentation and medical image classification, and specifically to a deep fusion learning-assisted analysis method for demyelination in children's brain MRI. Background Art

[0002] Central Nervous System Demyelinating Diseases (CNSD) are a group of diseases characterized by destruction or loss of myelin in the brain and spinal cord, which are common in children and adolescents. The pathological mechanism of CNSD is related to widespread demyelinating lesions caused by white matter damage. Lesions in the white matter of the brain, spinal cord and brainstem, cerebellum and optic nerve are the main causes of clinical symptoms and neurological signs in patients. Severe patients have a high incidence of complications and many sequelae. Multiple sclerosis (MS), acute disseminated encephalomyelitis (ADEM), and neuromyelitis optica spectrum disorders (NMOSD) are common CNS demyelinating diseases. ADEM and NMOSD are both primary CNS demyelinating diseases, with similarities in their first symptoms, neurological symptoms and signs, and clinical diagnosis is somewhat difficult.

[0003] In recent years, magnetic resonance imaging (MRI) diagnostic technology has developed rapidly, and MRI has played an increasingly important role in the diagnosis, treatment and prognosis of demyelinating diseases of the central nervous system. MRI can detect lesions in the white matter of the brain. For the identification of demyelinating diseases, manual segmentation of high-signal white matter is the gold standard for lesion identification. However, this method is not only time-consuming, but also requires expert experience. In the past few years, deep learning methods have achieved remarkable success in the field of detecting brain lesions, such as Alzheimer's disease (AD), stroke, brain tumors, etc. Different studies have shown that deep learning methods can be successfully applied to medical image segmentation, disease detection and classification, but currently classification and segmentation are usually regarded as two independent steps. On the one hand, in brain MRI, there are mainly three tissue regions: white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF). The lesion area occurs in the white matter. Since the brain of children is not fully developed, this further increases the difficulty of identifying the disease. The importance of different tissue regions cannot be weighed in a single segmentation-based method. On the other hand, a single classification method cannot provide pixel-level information, that is, the specific size and range of the lesion in the image cannot be obtained. Therefore, this patent proposes a deep fusion learning-based auxiliary analysis method for demyelination in children's brain MRI. The innovations of this method are as follows: 1) By constructing a deep learning network model to simultaneously perform segmentation and classification tasks on brain MRI images, the patient's disease and lesion area can be determined. The segmentation-based method can provide pixel-level information image organizational structure information, such as size and position, which will be more conducive to image classification; the classification-based method can identify the area of ​​interest inside the image, that is, the white matter area. This will promote the accuracy of segmentation. The feature fusion of the two methods can not only promote segmentation tasks but also classification tasks. 2) Childhood demyelination auxiliary analysis system. The system integrates and visualizes the proposed auxiliary analysis method.

[0004] The algorithm and system proposed in this patent will greatly shorten the patient's diagnosis time and diagnosis cost, enable patients to receive timely treatment, alleviate the neurological deficits left over from the patient's prognosis process, and obtain better prognosis results. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a deep fusion learning-based demyelination-assisted analysis method for children's brain MRI, which includes the construction of a deep learning network model and algorithm design, and the design of a children's demyelination-assisted analysis system.

[0006] To achieve the above object, the technical solution of the present invention mainly includes the following steps:

[0007] Step 1: Preprocessing of childhood demyelinating disease data and construction of dataset.

[0008] The brain MRI images in the dataset are obtained from hospitals and are all real patient data. After obtaining the data, the data is first cleaned, and then the data information is classified, segmented and labeled to complete the construction of the childhood demyelinating disease dataset.

[0009] Step 2: Deep fusion learning to assist in the design of demyelinating disease model.

[0010] The model consists of three modules:

[0011] 1) Shared module: The input brain MRI image is downsampled by convolution and pooling to obtain the image features and compress the features to obtain the final deep feature map.

[0012] 2) Segmentation module: The input of the segmentation module is the output of the shared module. Deconvolution is used for upsampling, and the feature information corresponding to the shared module is spliced. Finally, two convolution operations are performed to output the high signal white matter segmentation map (Mask output ).

[0013] 3) Classification module: The input of the classification module is the output of the shared module. The feature map is downsampled by convolution and pooling. Finally, the feature map classification label (Label) is output through average pooling and full connection operations. output ).

[0014] The model uses a joint loss function: the loss functions of the two tasks of segmentation and classification are combined.

[0015] Step 3: Train the deep fusion learning demyelinating disease auxiliary model using the constructed pediatric demyelinating disease dataset.

[0016] Step 4: Use the trained deep fusion learning demyelinating disease auxiliary model to complete the acquisition and feature classification of the high-signal white matter segmentation map, and display the high-signal white matter segmentation map and feature classification labels through the visualization result model.

[0017] The specific steps of step 1 are as follows:

[0018] 1-1. Data cleaning: First, screen the collected data and remove invalid, unclear, artifact-containing and other abnormal data.

[0019] 1-2. Data annotation:

[0020] Each image contains two types of annotation information: high signal white matter segmentation map and feature classification label.

[0021] 1) High signal white matter segmentation map annotation. Under the guidance of the doctor, the data was annotated using the professional annotation tool 3Dslicer. The high signal white matter segmentation standard map (Mask true ).

[0022] 2) Feature map classification label annotation. By sorting out the case information, the images in the dataset are classified into the following three categories: 1) Brain MRI data without brain abnormalities. 2) Brain MRI data of patients with acute disseminated encephalomyelitis. 3) Brain MRI data of patients with neuromyelitis optica spectrum disorder. Obtain the feature map classification label (Label true ).

[0023] 1-3. Data enhancement: Change the size of each image and unify all data to a fixed size; normalize the data and map each pixel of each image to the range of 0 to 1 to improve the convergence speed of the model.

[0024] The specific steps of step 2 are as follows:

[0025] 2-1. The shared module is the input for subsequent segmentation and classification. The shared module includes a total of 5 blocks, each of which contains 2 convolutional layers and 1 pooling layer. Batch normalization and activation function ReLU are performed after each convolution layer. The input brain MRI image is downsampled through convolution and pooling operations to obtain the image features and compress the features. After the image is operated by 5 blocks, a deep feature map is obtained, the size of which is 32*32*512 (length*width*number of convolution kernels). The convolution operation (Convolution) is to perform a convolution operation after the image is input into the model, using a 3*3 convolution, and the convolution method is "SAME", so after the convolution operation, the length and width of the image are the same as the input, but the number of channels will change according to the number of convolution kernels. In the 5 blocks, the 2 convolution layers in each block are the same, and the 5 blocks use 32, 64, 128, 256, and 512 convolution kernels respectively. For each pixel p on the output feature map y after each block. 0 , the convolution expression is:

[0026]

[0027] where p n is an enumeration of positions listed in R = (-1, -1), (0, -1), ..., (0, 1), (1, 1), x(p 0 +p n ) represents pixel p 0 +p n The value on . Represents the value of pixel p0. The size of the input feature map is W i ×W i , the size of the convolution kernel is F×F, the padding is P, and the stride is S. The size of the output feature map is calculated as:

[0028]

[0029] Where W represents the size of the output feature map. After the convolution layer, batch normalization and activation function ReLU are used to improve the generalization ability of the network. The formula of ReLU is as follows:

[0030]

[0031] After ReLU, the pooling operation follows: after the maximum pooling layer, the size is reduced to half of the original size.

[0032] 2-2. The input of the segmentation module is the output of the shared module. The segmentation module includes a total of 5 blocks. Each of the first 4 blocks contains 2 convolutional layers and 1 deconvolutional layer operations. Batch normalization and activation function ReLU are performed after each convolutional layer. The feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed, that is, the feature map output from the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module. The last block contains 2 1*1 convolutional layers. After the last block, the high signal white matter segmentation map (Mask output ). The image feature information of 64*64*256 in the sharing module is fused into the image of 64*64*512 in the segmentation module. After being spliced ​​together, the size is 64*64*768, and then feature extraction is continued.

[0033] Deconvolution is the process of mapping from low resolution to high resolution. It is used to expand the image size during upsampling and restore the image size.

[0034] 2-3. The input of the classification module is the output of the shared module. The classification module includes 4 blocks in total. The operations of the first two blocks are the same as those of the shared module, including 2 convolutional layers and 1 pooling layer, but the number of convolution kernels is different, 1024 and 2048 respectively. The third block is the average pooling layer, and the fourth block is the fully connected layer. After passing through the four blocks, the output feature map classification label (Label output ), where output 1 is an image of a normal brain, output 2 is an image of acute disseminated encephalomyelitis, and output 3 is a image of neuromyelitis optica spectrum disorder.

[0035] 2-4. The loss function in model training is calculated using the following formula:

[0036]

[0037] where λ 1 ,λ 2 are weights, and their purpose is to balance the contributions of the two tasks, classification and segmentation, in the joint loss. and represent segmentation and classification losses respectively. It is a loss function composed of a binary cross entropy loss function and a Sigmoid function. Its calculation formula is as follows:

[0038]

[0039] Where σ(x n ) is the Sigmoid function, which is used to map x to the interval (0,1):

[0040]

[0041] is the cross entropy loss function, which is calculated as follows:

[0042]

[0043] Where class is the label.

[0044] Step 3:

[0045] After completing the model design, set the hyperparameters to train the network model.

[0046] Hyperparameters: When training with the constructed pediatric demyelinating disease dataset, the images in the dataset are randomly divided into two parts with a ratio of 8:2, of which 80% are training samples and the remaining 20% ​​are test samples. SGD is used as the optimizer, the batch size is set to 16, and the initial learning rate is set to 10 -3 , decays in each epoch, and the minimum learning rate is set to 10 -5 During the experiment, it takes an average of 50 epochs to converge.

[0047] The specific steps of step 4 are as follows:

[0048] The visualization result model is a GUI interface designed by pyqt, which is used to segment the high-signal white matter area and display the feature map classification label. The sample images of the actual test are input into the model trained in step 2 to obtain the output result: the high-signal white matter segmentation map (Mask test) and feature classification labels (Label test ).

[0049] A deep fusion learning-based demyelination-assisted analysis system for children's brain MRI, including a sharing module, a segmentation module, a classification module, and a visualization result model.

[0050] Shared module: Use convolution and pooling to downsample the input brain MRI image to obtain the image features and compress the features to obtain the final deep feature map.

[0051] Segmentation module: The input of the segmentation module is the output of the shared module. Deconvolution is used for upsampling, and the feature information corresponding to the shared module is spliced. Finally, two convolution operations are performed to output the high signal white matter segmentation map (Mask output ).

[0052] Classification module: The input of the classification module is the output of the shared module. It uses convolution and pooling to downsample the feature map. Finally, it outputs the feature map classification label (Label) through average pooling and full connection operations. output ).

[0053] The visualization result model is used to display the high-signal white matter segmentation map output by the segmentation module and the feature classification label output by the classification module.

[0054] Furthermore, the shared module is the input for subsequent segmentation and classification. The shared module includes a total of 5 blocks, each of which contains 2 convolutional layers and 1 pooling layer. Batch normalization and activation function ReLU are performed after each convolutional layer. The input brain MRI image is downsampled through convolution and pooling operations to obtain the image features and compress the features. After the image is operated by 5 blocks, a deep feature map is obtained, the size of which is 32*32*512 (length*width*number of convolution kernels). The convolution operation (Convolution) is to perform a convolution operation after the image is input into the model, using a 3*3 convolution, and the convolution method is "SAME", so after the convolution operation, the length and width of the image are the same as the input, but the number of channels will change according to the number of convolution kernels. In the 5 blocks, the 2 convolution layers in each block are the same, and the 5 blocks use 32, 64, 128, 256, and 512 convolution kernels respectively. For each pixel p on the output feature map y after each block. 0 , the convolution expression is:

[0055]

[0056] where p nis an enumeration of positions listed in R = (-1, -1), (0, -1), ..., (0, 1), (1, 1), x(p 0 +p n ) represents pixel p 0 +p n The value on . Represents pixel p 0 The size of the input feature map is W i ×W i , the size of the convolution kernel is F×F, the padding is P, and the stride is S. The size of the output feature map is calculated as:

[0057]

[0058] Where W represents the size of the output feature map. After the convolution layer, batch normalization and activation function ReLU are used to improve the generalization ability of the network. The formula of ReLU is as follows:

[0059]

[0060] After ReLU, the pooling operation follows: after the maximum pooling layer, the size is reduced to half of the original size.

[0061] Furthermore, the input of the segmentation module is the output of the shared module. The segmentation module includes a total of 5 blocks. Each of the first 4 blocks contains 2 convolutional layers and 1 deconvolutional layer operations. Batch normalization and activation function ReLU are performed after each convolutional layer. The feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed, that is, the feature maps output from the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module. The last block contains 2 1*1 convolutional layers, and the high signal white matter segmentation map (Mask) is output after the last block. output ). The image feature information of 64*64*256 in the sharing module is fused into the image of 64*64*512 in the segmentation module. After being spliced ​​together, the size is 64*64*768, and then feature extraction is continued.

[0062] Deconvolution is the process of mapping from low resolution to high resolution. It is used to expand the image size during upsampling and restore the image size.

[0063] Furthermore, the input of the classification module is the output of the shared module. The classification module includes 4 blocks in total. The operations of the first two blocks are the same as those of the shared module, including 2 convolutional layers and 1 pooling layer, but the number of convolution kernels is different, 1024 and 2048 respectively. The third block is the average pooling layer, and the fourth block is the fully connected layer. After passing through the four blocks, the feature map classification label (Label output ), where output 1 is an image of a normal brain, output 2 is an image of acute disseminated encephalomyelitis, and output 3 is a image of neuromyelitis optica spectrum disorder.

[0064] Furthermore, the visualization result model is a GUI interface designed by pyqt, which is used to segment high-model white matter areas and display feature map classification labels.

[0065] The beneficial effects of the present invention are as follows:

[0066] The present invention aims at the difficulty of segmentation and classification of demyelinating diseases in children, especially for young doctors, and proposes a method for auxiliary analysis of demyelinating in children's brain MRI by deep fusion learning. The algorithm simultaneously performs two tasks of segmentation and classification on brain MRI images through a deep learning method to determine the size, position and feature map classification of the lesion area. The segmentation-based method can provide pixel-level information image organizational structure information, such as size and position, which will be more conducive to the feature map classification of the image; the feature map classification-based method can identify the area of ​​interest inside the image, that is, the white matter area. This will promote the accuracy of high-signal white matter segmentation. The feature fusion of the two methods can not only promote the high-signal white matter segmentation task but also the feature map classification task. For clinical use, this can not only assist doctors in clinical diagnosis, but also greatly improve the rate of image annotation, which will greatly shorten the patient's diagnosis time and diagnosis cost, enable patients to obtain timely treatment, alleviate the neurological deficits left over from the patient's prognosis process, and obtain better prognosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a flow chart of an embodiment of the present invention;

[0068] Figure 2 This is the high signal white matter segmentation effect diagram;

[0069] Figure 3 Schematic diagram of the visualization result model. DETAILED DESCRIPTION

[0070] The present invention is further described in detail below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto.

[0071] The following combination Figure 1 The implementation method of the present invention is further described.

[0072] Step 1. Data preprocessing and data set construction of pediatric demyelinating diseases: By collecting data in hospitals, we collected multiple head posture MRI image sequences of 310 pediatric patients, including T1, T2, FLAIR, DWI and other sequences, totaling more than 40,000 images in dicom format. The brain MRI images in the database are all real patient data, including healthy brains and pediatric demyelinating diseases, including acute disseminated encephalomyelitis, neuromyelitis optica spectrum disorders, etc.

[0073] 1-1. The original data set mainly selected and annotated the dicom images of the 3.0T head axial FLAIR sequence. The original data set contains 293 sets of demyelinating disease FLAIR sequences, a total of 3009 images and 150 sets of normal brain FLAIR sequences, a total of 1503 images. A total of 443 sets of MRI-FLAIR sequences and 4512 images.

[0074] 1-2. Segmentation and annotation: Use 3D Slicer annotation software to annotate the high-order white matter area and save it as a .nrrd file. Each sequence has 11 images. After the annotation is completed, we separate each image and name it in the form of a serial number plus an underscore plus a serial number.

[0075] Take "M066604.nrrd" as an example: "M066604_9.nrrd", "M066604_7.nrrd", "M066604_6.nrrd", "M066604_2.nrrd", "M066604_1.nrrd", "M066604_0.nrrd".

[0076] 1-3. Classification labeling: Extract category information from the hospital's case information, label the image without brain abnormality as 1, acute disseminated encephalomyelitis as 2, and neuromyelitis optica spectrum disorder as 3. Store the information in a txt file to provide data information support for subsequent network training.

[0077] Step 2: Deep fusion learning demyelinating disease auxiliary model design:

[0078] The model consists of three modules:

[0079] 1) Shared module: The module includes 4 blocks in total, each of which contains 2 convolutional layers and 1 pooling layer. The input brain MRI image is downsampled by convolution and pooling to obtain and compress the image features. After the image is operated by 4 blocks, a deep feature map is obtained with a size of 32*32*512 (length*width*number of convolution kernels).

[0080] 2) Segmentation module: The input of the segmentation module is the output of the shared module. This module includes a total of 5 blocks. Each of the first 4 blocks contains 2 convolutional layers and 1 upsampling operation. Batch normalization and activation function ReLU are performed after each convolutional layer. The feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed. That is, the feature map output from the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module. The last block contains 2 1*1 convolutional layers. After the last block, the high signal white matter segmentation map (Mask output ).

[0081] 3) Classification module: The input of the classification module is the output of the shared module. There are 4 blocks in this module. The operations of the first two blocks are the same as those of the shared module, including 2 convolutional layers and 1 pooling layer. Batch normalization and activation function ReLU are performed after each convolutional layer, but the number of convolution kernels is different, 1024 and 2048 respectively. The third block is the average pooling layer, and the fourth block is the fully connected layer. After the four blocks, the feature map classification label (Label) is output. output ).

[0082] The model uses a joint loss function: the loss functions of the two tasks of segmentation and classification are combined.

[0083] Model training: After completing the model design, set hyperparameters to train the network model.

[0084] 2-1. The shared module is the input for subsequent segmentation and classification. The shared module includes a total of 5 blocks, each of which contains 2 convolutional layers and 1 pooling layer. Batch normalization and activation function ReLU are performed after each convolution layer. The input brain MRI image is downsampled through convolution and pooling operations to obtain the image features and compress the features. After the image is operated by 5 blocks, a deep feature map is obtained, the size of which is 32*32*512 (length*width*number of convolution kernels). The convolution operation (Convolution) is to perform a convolution operation after the image is input into the model, using a 3*3 convolution, and the convolution method is "SAME", so after the convolution operation, the length and width of the image are the same as the input, but the number of channels will change according to the number of convolution kernels. In the 5 blocks, the 2 convolution layers in each block are the same, and the 5 blocks use 32, 64, 128, 256, and 512 convolution kernels respectively. For each pixel p on the output feature map y after each block. 0 , the convolution expression is:

[0085]

[0086] where p n is an enumeration of positions listed in R = (-1, -1), (0, -1), ..., (0, 1), (1, 1), x(p 0 +p n ) represents pixel p 0 +p n The value on . Represents the value of pixel p0. The size of the input feature map is W i ×W i , the size of the convolution kernel is F×F, the padding is P, and the stride is S. The size of the output feature map is calculated as:

[0087]

[0088] Where W represents the size of the output feature map. After the convolution layer, batch normalization and activation function ReLU are used to improve the generalization ability of the network. The formula of ReLU is as follows:

[0089]

[0090] After ReLU, the pooling operation follows: after the maximum pooling layer, the size is reduced to half of the original size.

[0091] 2-2. The input of the segmentation module is the output of the shared module. The segmentation module includes a total of 5 blocks. Each of the first 4 blocks contains 2 convolutional layers and 1 deconvolutional layer operations. Batch normalization and activation function ReLU are performed after each convolutional layer. The feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed, that is, the feature map output from the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module. The last block contains 2 1*1 convolutional layers. After the last block, the high signal white matter segmentation map (Mask output ),like Figure 1 The image feature information of 64*64*256 in the sharing module is fused into the image of 64*64*512 in the segmentation module, and the size of the image after splicing is 64*64*768, and then feature extraction is continued. Figure 2 This is the high signal white matter segmentation effect diagram;

[0092] Deconvolution is the process of mapping from low resolution to high resolution. It is used to expand the image size during upsampling and restore the image size.

[0093] 2-3. The input of the classification module is the output of the shared module. The classification module includes 4 blocks in total. The operations of the first two blocks are the same as those of the shared module, including 2 convolutional layers and 1 pooling layer, but the number of convolution kernels is different, 1024 and 2048 respectively. The third block is the average pooling layer, and the fourth block is the fully connected layer. After passing through the four blocks, the output feature map classification label (Label output ), where output 1 is an image of a normal brain, output 2 is an image of acute disseminated encephalomyelitis, and output 3 is a image of neuromyelitis optica spectrum disorder.

[0094] 2-4. The loss function in model training is calculated using the following formula:

[0095]

[0096] where λ 1 ,λ 2 are weights, and their purpose is to balance the contributions of the two tasks, classification and segmentation, in the joint loss. and represent segmentation and classification losses respectively. It is a loss function composed of a binary cross entropy loss function and a Sigmoid function. Its calculation formula is as follows:

[0097]

[0098] Where σ(x n ) is the Sigmoid function, which is used to map x to the interval (0,1):

[0099]

[0100] is the cross entropy loss function, which is calculated as follows:

[0101]

[0102] Where class is the label.

[0103] Step 3: Train the deep fusion learning demyelinating disease auxiliary model using the constructed pediatric demyelinating disease dataset.

[0104] After completing the model design, set the hyperparameters to train the network model.

[0105] Hyperparameters: When training with the constructed pediatric demyelinating disease dataset, the images in the dataset are randomly divided into two parts with a ratio of 8:2, of which 80% are training samples and the remaining 20% ​​are test samples. SGD is used as the optimizer, the batch size is set to 16, and the initial learning rate is set to 10 -3, decays in each epoch, and the minimum learning rate is set to 10 -5 During the experiment, it takes an average of 50 epochs to converge.

[0106] Step 4: Use the trained deep fusion learning demyelinating disease auxiliary model to complete the acquisition and feature classification of the high-signal white matter segmentation map, and display the high-signal white matter segmentation map and feature classification labels through the visualization result model.

[0107] In order to more conveniently view the high signal white matter segmentation and feature map classification results of demyelinating disease in children of the present invention, a GUI interface is designed through pyqt. Select the test image and input the image into the model we pre-trained in step 2 to obtain the high signal white matter segmentation map (Mask output ) and feature classification results (Label output ). Figure 3 Schematic diagram of the visualization result model.

Claims

1. A deep fusion learning-based demyelination-assisted analysis method for children's brain MRI, characterized in that: Here are the steps: Step 1: Data preprocessing and dataset construction for pediatric demyelinating diseases; The brain MRI images in the dataset are obtained from hospitals and are all real patient data. After obtaining the data, the data is first cleaned, and then the data information is classified, segmented and labeled to complete the construction of the childhood demyelinating disease dataset. Step 2: Deep fusion learning demyelinating disease auxiliary model design; The model consists of three modules: 1) Shared module: The input brain MRI image is downsampled by convolution and pooling to obtain the image features and compress the features to obtain the final deep feature map; 2) Segmentation module: The input of the segmentation module is the output of the shared module; deconvolution is used for upsampling, and feature splicing is performed with the feature information corresponding to the shared module, and finally two convolution operations are performed to output the high signal white matter segmentation map; 3) Classification module: The input of the classification module is the output of the shared module. The feature map is downsampled by convolution and pooling. Finally, the feature map classification label is output through average pooling and full connection operations. The model uses a joint loss function: the loss functions of the two tasks of segmentation and classification are combined; Step 3: Train the deep fusion learning demyelinating disease auxiliary model using the constructed pediatric demyelinating disease dataset; Step 4: Use the trained deep fusion learning demyelinating disease auxiliary model to complete the acquisition of high-signal white matter segmentation map and feature classification, and use the visualization result model to display the high-signal white matter segmentation map and feature classification labels; The specific steps of step 2 are as follows: 2-1. The shared module is the input for subsequent segmentation and classification. The shared module includes 5 blocks in total, each of which contains 2 convolutional layers and 1 pooling layer. Batch normalization and activation function ReLU are performed after each convolutional layer. The input brain MRI image is downsampled by convolution and pooling operations to obtain the image features and compress the features. After the image is operated by 5 blocks, a deep feature map with a size of 32*32*512 is obtained. The convolution operation is to perform a convolution operation after the image is input into the model, using a 3*3 convolution, and the convolution mode is "SAME". Therefore, after the convolution operation, the length and width of the image are the same as the input, but the number of channels will change according to the number of convolution kernels. In the 5 blocks, the 2 convolution layers in each block are the same, and the 5 blocks use 32, 64, 128, 256, and 512 convolution kernels respectively. For each pixel p0 on the output feature map y after each block, the convolution expression is: where p n is an enumeration of positions listed in R = (-1, -1), (0, -1), ..., (0, 1), (1, 1), x(p0+p n ) represents pixel p0+p n The value on Represents the value of pixel p0; the size of the input feature map is W i ×W i , the size of the convolution kernel is F×F, the padding is P, and the stride is S; the size of the output feature map is calculated as: Where W represents the size of the output feature map; after the convolution layer, batch normalization and activation function ReLU are used to improve the generalization ability of the network; the formula of ReLU is as follows: After ReLU, the next step is pooling: after the maximum pooling layer, the size is reduced to half of the original size; 2-2. The input of the segmentation module is the output of the shared module. The segmentation module includes 5 blocks in total. Each of the first 4 blocks contains 2 convolutional layers and 1 deconvolutional layer operations. Batch normalization and activation function ReLU are performed after each convolutional layer. The feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed, that is, the feature maps output from the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module. The last block contains 2 1*1 convolutional layers, and a high-signal white matter segmentation map is output after the last block. The 64*64*256 image feature information in the shared module is fused into the 64*64*512 image in the segmentation module. After splicing together, the size is 64*64*768, and then feature extraction is continued. Deconvolution is the process of mapping from low resolution to high resolution. It is used to expand the image size during upsampling and restore the image size. 2-3. The input of the classification module is the output of the shared module. The classification module includes 4 blocks in total. The operations of the first two blocks are the same as those of the shared module, including 2 convolutional layers and 1 pooling layer, but the number of convolution kernels is different, 1024 and 2048 respectively. The third block is the average pooling layer, and the fourth block is the fully connected layer. After 4 blocks, the feature map classification label is output, where output 1 is an image without abnormalities in the brain, output 2 is acute disseminated encephalomyelitis, and output 3 is neuromyelitis optica spectrum disorder. 2-4. The loss function in model training is calculated using the following formula: Where λ1, λ2 are weights, their purpose is to balance the contribution of the two tasks of classification and segmentation in the joint loss; and Represent segmentation and classification losses respectively; It is a loss function composed of a binary cross entropy loss function and a Sigmoid function. Its calculation formula is as follows: Where σ(x n ) is the Sigmoid function, which is used to map x to the interval (0, 1): is the cross entropy loss function, which is calculated as follows: Where class is the label.

2. The method for analyzing demyelination in children's brain MRI using deep fusion learning according to claim 1, characterized in that: The specific steps of step 1 are as follows: 1-1. Data cleaning: First, screen the collected data and remove invalid, unclear, artifact-containing and other abnormal data; 1-2. Data annotation: Each image contains two types of annotation information: high signal white matter segmentation map and feature classification label; 1) High signal white matter segmentation map annotation: Under the guidance of the doctor, the data is annotated using the professional annotation tool 3Dslicer to obtain the high signal white matter segmentation standard map; 2) Feature map classification label annotation: By sorting out case information, the images in the dataset are classified into the following three categories: 1) Brain MRI data without brain abnormalities; 2) Brain MRI data of patients with acute disseminated encephalomyelitis; 3) Brain MRI data of patients with neuromyelitis optica spectrum disorder; obtain feature map classification labels; 1-3. Data enhancement: Change the size of each image and unify all data to a fixed size; normalize the data and map each pixel of each image to the range of 0 to 1 to improve the convergence speed of the model.

3. The method for assisting analysis of demyelination in children's brain MRI by deep fusion learning according to claim 2, characterized in that: Step 3: After completing the model design, set the hyperparameters to train the network model; Hyperparameters: When training with the constructed pediatric demyelinating disease dataset, the images in the dataset are randomly divided into two parts with a ratio of 8:2, of which 80% are training samples and the remaining 20% ​​are test samples; SGD is used as the optimizer, the batch size is set to 16, and the initial learning rate is set to 10 -3 , decays in each epoch, and the minimum learning rate is set to 10 -5 ; During the experiment, convergence takes an average of 50 epochs.

4. The method for analyzing demyelination in children's brain MRI using deep fusion learning according to claim 3 is characterized in that: The specific steps of step 4 are as follows: The visualization result model is a GUI interface designed by pyqt, which is used to segment high-signal white matter areas and display feature map classification labels; the sample images of actual tests are input into the model trained in step 2 to obtain the output results: high-signal white matter segmentation maps and feature classification labels.

5. A deep fusion learning-based children's brain MRI demyelination auxiliary analysis system, characterized in that: Includes sharing module, segmentation module, classification module and visualization result model; Shared module: Use convolution and pooling to downsample the input brain MRI image to obtain the image features and compress the features to obtain the final deep feature map; Segmentation module: The input of the segmentation module is the output of the shared module; deconvolution is used for upsampling, and feature splicing is performed with the feature information corresponding to the shared module, and finally two convolution operations are performed to output the high signal white matter segmentation map; Classification module: The input of the classification module is the output of the shared module. It uses convolution and pooling to downsample the feature map. Finally, it outputs the feature map classification label through average pooling and full connection operations. The visualization result model is used to display the high-signal white matter segmentation map output by the segmentation module and the feature classification label output by the classification module; The shared module is the input for subsequent segmentation and classification; the shared module includes a total of 5 blocks, each of which contains 2 convolutional layers and 1 pooling layer, and batch normalization and activation function ReLU are performed after each convolutional layer; the input brain MRI image is downsampled by convolution and pooling operations to obtain the image features and compress the features; after the image is operated by 5 blocks, a deep feature map with a size of 32*32*512 is obtained; the convolution operation is to perform a convolution operation after the image is input into the model, using a 3*3 convolution, and the convolution mode is "SAME", so after the convolution operation, the length and width of the image are the same as the input, but the number of channels will change according to the number of convolution kernels. In the 5 blocks, the 2 convolution layers in each block are the same, and the 5 blocks use 32, 64, 128, 256, and 512 convolution kernels respectively; for each pixel p0 on the output feature map y after each block, the convolution expression is: where p n is an enumeration of positions listed in R = (-1, -1), (0, -1), ..., (0, 1), (1, 1), x(p0+p n ) represents pixel p0+p n The value on Represents the value of pixel p0; the size of the input feature map is W i ×W i , the size of the convolution kernel is F×F, the padding is P, and the stride is S; the size of the output feature map is calculated as: Where W represents the size of the output feature map; after the convolution layer, batch normalization and activation function ReLU are used to improve the generalization ability of the network; the formula of ReLU is as follows: After ReLU, the pooling operation follows: after the maximum pooling layer, the size is reduced to half of the original size.

6. The deep fusion learning children's brain MRI demyelination auxiliary analysis system according to claim 5 is characterized in that: The input of the segmentation module is the output of the shared module; the segmentation module includes a total of 5 blocks, each of the first 4 blocks contains 2 convolutional layers and 1 deconvolutional layer operations, and batch normalization and activation function ReLU are performed after each convolutional layer; the feature information corresponding to the shared module is added to each corresponding block, and feature splicing is performed, that is, the feature maps output in the first 4 blocks in the shared module and the corresponding image size are spliced ​​into the segmentation module; the last block contains 2 1*1 convolutional layers, and a high signal white matter segmentation map is output after the last block; the image feature information of 64*64*256 in the shared module is fused into the image of 64*64*512 in the segmentation module, and the size of the spliced ​​together is 64*64*768, and then feature extraction is continued; Deconvolution is the process of mapping from low resolution to high resolution. It is used to expand the image size during upsampling and restore the image size.

7. The deep fusion learning children's brain MRI demyelination auxiliary analysis system according to claim 6 is characterized in that: The input of the classification module is the output of the shared module; the classification module includes a total of 4 blocks; the operations of the first two blocks are consistent with the blocks in the shared module, including 2 convolutional layers and 1 pooling layer, but the number of convolution kernels is different, 1024 and 2048 respectively; the third block is the average pooling layer, and the fourth block is the fully connected layer; after passing through 4 blocks, the feature map classification label is output, where output 1 is an image without abnormalities in the brain, output 2 is acute disseminated encephalomyelitis, and output 3 is neuromyelitis optica spectrum disorder.

8. The deep fusion learning children's brain MRI demyelination auxiliary analysis system according to claim 7 is characterized in that: The visualization result model is a GUI interface designed by pyqt, which is used to segment high-resolution white matter areas and display feature map classification labels.

Images