Automatic Brain Region Segmentation Method and Device Based on PET / MR Imaging System

Abstract

The present invention relates to an automatic brain region segmentation method and device based on a PET / MR imaging system, which fuses MRI and PET image features to improve the precision and accuracy of brain region segmentation. The method uses MRI and PET images of the same individual as a sample; after removing the skull and drawing labels on the MRI image in each sample, the MRI image is used as a template to register the PET image, so that the PET and MRI images share labels; the registered image is sliced according to cross-sections, and the sliced images are subjected to data normalization and one-hot encoding of the labels; a Unet model with two input channels and one output channel is established; the slices of the MRI and PET images are used as input to the Unet model to obtain brain region segmentation results after feature fusion. By fusing the features of the PET / MR dual modality, the present invention retains individual specificity and improves the precision and accuracy of brain region segmentation.

Description

Technical Field

[0001] The present disclosure relates to medical image segmentation, and in particular to a method and device for automatic brain region segmentation based on a PET / MR imaging system. Background Art

[0002] Integrated positron emission tomography (PET) / magnetic resonance imaging (MRI) is a novel multimodal imaging system that integrates PET and MRI. It enables simultaneous acquisition of data from two different devices within the same space, combining the high-resolution soft tissue imaging and multi-parameter, multifunctional imaging capabilities of an MRI system with the high sensitivity of radiotracer metabolism and quantitative data analysis of a PET system. Accurate brain segmentation significantly impacts clinical diagnosis and is of great value in the diagnosis of cerebrovascular disease, Alzheimer's disease, epilepsy, Parkinson's disease, neurodegenerative diseases, as well as in the research of neuropsychiatric drugs and brain function.

[0003] Traditional methods and segmentation tools use a standard brain template approach, ignoring individual differences. Most algorithms are based on the segmentation of a few brain regions or brain tumors. These algorithms are not robust when applied to the entire brain. Currently, because PET resolution is far lower than MRI, most brain segmentation algorithm research focuses on MRI, with few studies using PET for brain segmentation. However, PET is a functional imaging technique that focuses on cellular activity for case detection, making it highly valuable for brain segmentation. Summary of the Invention

[0004] In view of this, the main purpose of the present invention is to propose a method or device for automatic brain region segmentation based on a PET / MRI imaging system, which can retain individual specificity and has the respective advantages of PET images and MRI images, which is conducive to improving the overall segmentation accuracy.

[0005] In a first aspect, the present invention provides an automatic brain region segmentation method based on a PET / MRI imaging system, the method comprising the following steps:

[0006] S100, taking the MRI image and PET image of the same person as a sample; after removing the skull and drawing labels on the MRI image in each sample, using the MRI image as a template to perform registration processing on the PET image so that the PET image and the MRI image share the same label;

[0007] S200, slicing the registered image according to the cross section, performing data normalization on the slice image, and performing one-hot encoding on the label;

[0008] S300, establish a Unet model with two input channels;

[0009] S400: Using slices of the MRI image and the PET image as inputs to the Unet model to obtain a brain segmentation result after feature fusion.

[0010] Preferably, in the method, the Unet model is a 7-layer Unet model;

[0011] The 7-layer Unet model includes 14 convolution modules. In the first 6 convolution modules, a downsampling module is set after each convolution module. From the 8th to the 13th convolution module, an upsampling module is set before each convolution module.

[0012] The downsampling module uses maximum pooling to compress the input image;

[0013] The upsampling module uses inverse convolution to amplify the input image;

[0014] In the first 13 convolutional modules, each convolutional module has two convolutional layers for feature extraction, and a normalization layer and an activation layer are added to each convolutional layer in sequence;

[0015] The 14th convolutional module has only one convolutional layer, which is used to output various brain region segmentation results.

[0016] The normalization method adopted by the normalization layer is instance normalization, and the function of the activation layer selects the LeakyReLU activation function.

[0017] Preferably, in the method, the number of brain region labels is 43, including 42 brain region segmentation labels and 1 background.

[0018] Preferably, in the method, the 7-layer Unet model uses the following loss function to measure the effect of each model training. The smaller the loss function, the closer the brain region segmentation result of the current model for the PET image is to the value corresponding to the MRI image label;

[0019] Loss = αL de +βL ce

[0020]

[0021]

[0022] In the above formula:

[0023] α and β are the weights of the cross entropy loss function and the Dice loss function, respectively, both set to 1; M is the number of brain region segmentation categories; N is the total number of pixels on each slice;

[0024] p i，jis the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category;

[0025] g i，j It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

[0026] Preferably, in the method, the loss function is optimized using an Adam optimizer.

[0027] In a second aspect, the present invention proposes an automatic brain region segmentation device based on a PET / MR imaging system, the device comprising a preprocessing module and a Unet module;

[0028] The preprocessing module includes a registration unit, a slicing unit, a data normalization unit, and a one-hot encoding unit;

[0029] The registration unit takes the MRI image and PET image of the same person as a sample; after removing the skull and drawing labels on the MRI image in each sample, uses the MRI image as a template to perform registration processing on the PET image so that the PET image and the MRI image share the same label;

[0030] The slicing unit slices the registered image according to the cross section;

[0031] The data normalization unit is configured to perform normalization processing on the MRI slice image and the PET slice image;

[0032] The one-hot encoding unit: sets the position of the channel number corresponding to the label category to 1 and the other positions to 0;

[0033] The Unet module uses slices of MRI images and PET images as inputs of the Unet model to obtain brain region segmentation results after feature fusion.

[0034] Preferably, in the device, the Unet module is a 7-layer Unet model;

[0035] The 7-layer Unet model includes 14 convolution modules. In the first 6 convolution modules, a downsampling module is set after each convolution module. From the 8th to the 13th convolution module, an upsampling module is set before each convolution module.

[0036] The downsampling module uses maximum pooling to compress the input image;

[0037] The upsampling module uses inverse convolution to amplify the input image;

[0038] In the first 13 convolutional modules, each convolutional module has two convolutional layers for feature extraction, and a normalization layer and an activation layer are added to each convolutional layer in sequence;

[0039] The 14th convolutional module has only one convolutional layer, which is used to output brain segmentation results;

[0040] The normalization method adopted by the normalization layer is instance normalization, and the function of the activation layer selects the LeakyReLU activation function.

[0041] Preferably, in the device, the number of brain region labels is 43, including 42 brain region segmentation labels and 1 background.

[0042] Preferably, in the device, the 7-layer Unet model uses the following loss function to measure the effect of each model training. The smaller the loss function, the closer the brain region segmentation result of the current model for the PET image is to the value corresponding to the MRI image label;

[0043] Loss = αL de +βL ce

[0044]

[0045]

[0046] In the above formula:

[0047] α and β are the weights of the cross entropy loss function and the Dice loss function, respectively, both set to 1; M is the number of brain region segmentation categories; N is the total number of pixels on each slice;

[0048] p i，j is the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category;

[0049] g i，j It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

[0050] Preferably, in the device, the loss function is optimized using an Adam optimizer.

[0051] Compared with existing technologies:

[0052] The present invention preserves individual specificity by processing MRI images into templates; improves the generalization ability of the algorithm by using labels as model input; and improves the overall segmentation accuracy by fusing PET / MR features. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A schematic diagram of a Unet model in one embodiment of the present disclosure;

[0055] Among them: ① is the input of 2 convolutions; ② is the maximum pooling layer; ③ is the inverse convolution and ④ is the output after 1 convolution;

[0056] Figure 2 A cross-sectional slice of the coronal plane of all class labels corresponding to the gold standard in one embodiment of the present disclosure;

[0057] Figure 3 A cross-sectional slice of a coronal plane corresponding to a single-channel MRI image with all class labels in one embodiment of the present disclosure;

[0058] Figure 4 A cross-sectional slice of a coronal plane corresponding to a single-channel MRI image with all class labels in one embodiment of the present disclosure;

[0059] Figure 5 This is a cross-sectional slice of the coronal plane of all class labels corresponding to the PET / MR dual input in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0061] The terms "including" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or devices is not necessarily limited to those steps or devices expressly listed but may include other steps or devices not expressly listed or inherent to such process, method, product, or apparatus.

[0062] To make the purpose, technical solutions and advantages of this application clearer, the technical solutions of the present invention are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0063] In one embodiment, the novel multimodal imaging system used is an integrated positron emission tomography (PET) / magnetic resonance imaging (MRI), which integrates PET and MRI into one, enabling simultaneous acquisition of two different devices in the same space. It combines the soft tissue high-resolution and multi-parameter multifunctional imaging characteristics of the MRI system with the high sensitivity of radiotracer metabolism and data quantification characteristics of the PET system.

[0064] This system can acquire two images simultaneously. By adopting the following method, the features of MRI images and PET images can be integrated, which can improve the accuracy and precision of PET brain segmentation while preserving individual differences. The specific implementation steps are as follows:

[0065] S100, taking the MRI image and PET image of the same person as a sample; after removing the skull and drawing labels on the MRI image in each sample, using the MRI image as a template to perform registration processing on the PET image so that the PET image and the MRI image share the same label;

[0066] S200, slicing the registered image according to the cross section, performing data normalization on the slice image, and performing one-hot encoding on the label;

[0067] S300, establish a Unet model with two input channels;

[0068] S400: Using slices of the MRI image and the PET image as inputs to the Unet model to obtain a brain segmentation result after feature fusion.

[0069] Since different people have different brain structures, if a brain template is used, individual heterogeneity will be ignored. Therefore, in this embodiment, an MRI image of the same brain is used as a template to perform brain region segmentation on the PET image of the brain, preserving individual heterogeneity. The PET image after registration shares the same label with the MRI image. Specifically, when performing label processing in the MRI image, 42 labels and 1 background label are set in the brain region, and the PET image after registration shares these labels. Using PET and MR image slices as input to the 7-layer Unet model for training and learning can be used to improve the generalization ability of the model and improve the accuracy of brain region segmentation.

[0070] The 7-layer Unet model learns the features of PET molecular imaging and MRI soft tissue high resolution, and fuses the features of MRI and PET images to output brain segmentation results.

[0071] In a preferred embodiment, the Unet model is designed as a 7-layer Unet model, such as Figure 1 As shown. The left side of the model is used for feature extraction, and the right side is used for feature fusion. It should be understood that the terms "left", "right", etc. are only for the convenience of describing this application and simplifying the description, and cannot be understood as limiting this application. The 7-layer Unet model uses 14 convolution modules, 6 downsampling modules, and 6 upsampling modules. Specifically, in the first 6 convolution modules, a downsampling module is set after each convolution module; from the 8th convolution module to the 13th convolution module, an upsampling module is set in front of each convolution module. The downsampling module uses maximum pooling to compress the input image; the upsampling module uses inverse convolution to amplify the input image. In the first 13 convolution modules, each convolution module has two convolution layers for feature extraction. In order to accelerate the training of the neural network, accelerate the convergence speed and stability, a normalization layer is added after each convolution layer, and the normalization method is instance normalization. In order to speed up the convergence speed and solve the problem of gradient disappearance, an activation layer is added after the normalization layer, and the function of the activation layer selects the Leaky ReLU activation function. To facilitate the rapid calculation of the Dice loss for each class label, a convolutional module was added after the 13th convolutional module. This module consisted of a single convolutional layer, which output each brain region segmentation label as a corresponding channel. The number of output channels for this convolutional module was set to 43, including 42 class labels and one background label. The parameter settings for this 7-layer Unet model and the convolutional model are shown in Table 1.

[0072] Table 1

[0073] Module Kernel size step length filling Input Channel Output Channel The first convolutional module 3×3 1×1 1×1 2 32 The first downsampling module 2×2 1×1 - 32 32 The second convolution module 3×3 1×1 1×1 32 64 The second downsampling module 2×2 1×1 - 64 64 The third convolutional module 3×3 1×1 1×1 64 128 The third downsampling module 2×2 1×1 - 128 128 The fourth convolutional module 3×3 1×1 1×1 128 256 The fourth downsampling module 2×2 1×1 - 256 256 The fifth convolutional module 3×3 1×1 1×1 256 480 The fifth downsampling module 2×2 1×1 - 480 480 The 6th convolution module 3×3 1×1 1×1 480 480 The 6th downsampling module 2×2 1×1 - 480 480 The 7th convolutional module 3×3 1×1 1×1 480 480 The first upsampling module 2×1 2×1 - 480 480 The 8th convolutional module 3×3 1×1 1×1 960 480 The second upsampling module 2×2 1×1 - 480 480 The 9th convolutional module 3×3 1×1 1×1 960 480 The third upsampling module 2×2 1×1 - 480 256 The 10th convolutional module 3×3 1×1 1×1 512 256 The fourth upsampling module 2×2 1×1 - 256 128 The 11th convolutional module 3×3 1×1 1×1 256 128 The fifth upsampling module 2×2 1×1 - 128 64 The 12th convolutional module 3×3 1×1 1×1 128 64 The 6th upsampling module 2×2 1×1 - 64 32 The 13th convolutional module 3×3 1×1 1×1 64 32 The 14th convolutional module 1×1 1×1 - 32 43

[0074] As shown in the table above, in this embodiment, the left and right convolutional modules of each layer of the 7-layer Unet model each include a 3×3 convolution kernel. The left convolutional module is used to extract two image features for learning, and the right convolutional module is used to fuse the two image features. The right convolutional module of the first layer also includes a 1×1 convolutional kernel. Because the corresponding labels are one-hot encoded, the dice value for each label category can be easily and quickly calculated, thereby improving the brain segmentation speed of the 7-layer Unet model.

[0075] Figure 1In the figure, ① is the input for two convolutions; ② is the max pooling layer; ③ is the deconvolution, and ④ is the output for one convolution. In the left-hand side of the 7-layer Unet model, each layer's input first undergoes two convolutions in the convolution module for feature extraction, followed by downsampling in the downsampling module. According to the parameters in Table 1, MRI and PET images of size 256×192 are used as input. After feature information is extracted by the first convolution module and compressed by the downsampling module, the input for the second layer is reduced to 128×96. For the second to sixth layers on the left, after each layer undergoes convolution and downsampling processing, the resulting image sizes are 64×48, 32×24, 16×12, 8×6, and 4×6, respectively.

[0076] After convolution with the seventh convolutional module, the 4×6 image is upsampled by the upsampling module to enlarge the image. The enlarged image features and the feature image obtained from the sixth layer on the left are input to the convolutional module on the right, resulting in an input image of the fifth layer on the right, with an image size of 8×6. Next, from the fifth layer to the first layer on the right, each layer undergoes image enlargement by the upsampling module and convolution with the feature image of the corresponding layer, resulting in image sizes of 16×12, 32×24, 64×48, 128×96, and 256×192, respectively. Finally, convolution with the 14th convolutional module yields the segmentation results for 43 brain regions.

[0077] When training the 7-layer Unet model, the loss function used to optimize the model is as follows:

[0078] Loss = αL de +βL ce

[0079]

[0080]

[0081] In the above formula:

[0082] α and β are the weights of the cross entropy loss function and the Dice loss function, respectively, both set to 1; M is the number of brain region segmentation categories 43; N is the total number of pixels on each slice;

[0083] P i，j is the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category;

[0084] g i，j It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

[0085] The smaller the loss function, the closer the brain region segmentation result of the current model for the PET image is to the value corresponding to the MRI image label. Preferably, the loss function is optimized using the Adam optimizer.

[0086] In one embodiment, MRI images and PET images of a case are acquired simultaneously, and brain region segmentation is performed on the MRI images and PET images respectively using existing technologies, and the method of the present invention is used to acquire a brain region segmentation image. Figure 2-Figure 5 This is a coronal cross-sectional slice of all class labels. Figure 2 Schematic diagram of the gold standard. Figure 3 Schematic diagram of cross-sectional slices with only MRI image as input and corresponding output, Figure 4 Schematic diagram of cross-sectional slices with only PET image as input and corresponding output, Figure 5 This is a schematic diagram of cross-sectional slices in the present invention using MRI images and PET images as common input and corresponding output.

[0087] In one embodiment, the present invention implements an automatic brain region segmentation device based on a PET / MR imaging system. The device includes a preprocessing module and a Unet module. The preprocessing module includes a registration unit, a slicing unit, a data normalization unit, and a one-hot encoding unit. The registration unit treats MRI and PET images of the same individual as a sample. After removing the skull and labeling the MRI image in each sample, the MRI image is used as a template to register the PET image, ensuring that the PET and MRI images share the same label. The slicing unit slices the registered image according to cross-sections. The data normalization unit normalizes the MRI slices, PET slices, and labels. The one-hot encoding unit sets the channel number corresponding to the label category to 1 and all other channels to 0. The Unet module uses the slices of the MRI and PET images as input to the Unet model to obtain brain region segmentation results after feature fusion.

[0088] Preferably, the Unet module adopts a 7-layer Unet model; the 7-layer Unet model includes 14 convolution modules; in the first 6 convolution modules, a downsampling module is set after each convolution module; from the 8th convolution module to the 13th convolution module, an upsampling module is set in front of each convolution module; the downsampling module uses maximum pooling to reduce the input image; the upsampling module uses inverse convolution to enlarge the input image; in the first 13 convolution modules, each convolution module has two convolution layers for feature extraction, and a normalization layer and an activation layer are added to each convolution layer in sequence; the 14th convolution module has only one convolution layer for outputting the segmentation results of each brain region; the normalization method used in the normalization layer is instance normalization, and the function of the activation layer selects the Leaky ReLU activation function. There are 43 brain region labels, namely 42 brain region segmentation labels and 1 background. Specifically, the parameter settings of the 7-layer Unet model and the 1 convolution model are the same as Table 1.

[0089] Preferably, in the device, the 7-layer Unet model uses the following loss function to measure the effect of each model training. The smaller the loss function, the closer the brain region segmentation result of the current model for the PET image is to the value corresponding to the MRI image label;

[0090] Loss = αL de +βL ce

[0091]

[0092]

[0093] In the above formula: α, β are the weights of the cross entropy loss function and the Dice loss function, respectively, both set to 1; M is the number of brain region segmentation categories; N is the total number of pixels on each slice; p i，j is the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category; g i，j It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

[0094] Preferably, in the device, the loss function is optimized using an Adam optimizer.

[0095] Through the above description of the embodiments, those skilled in the art will clearly understand that the present disclosure can be implemented using software plus necessary general-purpose hardware. Of course, it can also be implemented using dedicated hardware, including application-specific integrated circuits, dedicated CPUs, dedicated memories, and dedicated components. Generally speaking, any function performed by a computer program can be easily implemented using corresponding hardware. Moreover, the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present disclosure, software implementation is often the preferred embodiment.

[0096] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields. The above-mentioned specific embodiments are merely illustrative and instructive, and are not restrictive. A person skilled in the art, guided by this specification and without departing from the scope of protection of the claims of the present invention, may also devise various forms, all of which fall within the scope of protection of the present invention.

Claims

1. An automatic brain region segmentation method based on a PET / MR imaging system, characterized in that: The method comprises the following steps: S100, taking the MRI image and PET image of the same person as a sample; S200, preprocessing each sample, the preprocessing comprising: removing the skull and drawing labels on the MRI image of each sample, and then using the image as a template to register the PET image so that the PET image and the MRI image share the same label; slicing the registered PET and MRI images according to cross-sections, performing data normalization on the MRI and PET slice images, and performing one-hot encoding on the labels; S300, establish a Unet model with two input channels; S400, using the preprocessed MRI image and PET image slices as inputs to the Unet model to obtain a brain segmentation result after feature fusion; When training the Unet model, the loss function used to optimize the model is as follows: In the above formula: α, β are the weights of the cross entropy loss function and the Dice loss function respectively; M is the brain region segmentation category; N is the total number of pixels on each slice; is the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category; It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

2. The method according to claim 1, characterized in that The Unet model is a 7-layer Unet model; The 7-layer Unet model includes 14 convolution modules. In the first 6 convolution modules, a downsampling module is set after each convolution module. From the 8th to the 13th convolution module, an upsampling module is set before each convolution module. The downsampling module uses maximum pooling to reduce the input image; The upsampling module uses inverse convolution to amplify the input image; In the first 13 convolutional modules, each convolutional module has two convolutional layers for feature extraction, and a normalization layer and an activation layer are added to each convolutional layer in sequence; The 14th convolutional module has only one convolutional layer, which is used to output brain segmentation labels; The normalization method adopted by the normalization layer is instance normalization, and the function of the activation layer selects the Leaky ReLU activation function.

3. The method according to claim 2, characterized in that There are 43 brain region labels, including 42 brain region segmentation labels and 1 background.

4. The method according to claim 1, wherein The loss function is optimized using the Adam optimizer.

5. An automatic brain region segmentation device based on a PET / MR imaging system, characterized in that: The device includes a preprocessing module and a Unet module; The preprocessing module includes a registration unit, a slicing unit, a data normalization unit, and a one-hot encoding unit; The registration unit is configured to treat the MRI image and the PET image of the same person as a sample; after removing the skull and drawing labels on the MRI image in each sample, use the MRI image as a template to perform registration processing on the PET image so that the PET image and the MRI image share the same label; The slicing unit is configured to slice the registered PET image and MRI image according to cross sections; The data normalization unit is configured to perform data normalization processing on the MRI slice image and the PET slice image; The one-hot encoding unit is configured to set the position where the channel number corresponding to the label category is located to 1 and the other positions to 0; The Unet module: takes slices of MRI images and PET images as inputs of the Unet model to obtain brain region segmentation results after feature fusion; When training the Unet model, the loss function used to optimize the model is as follows: In the above formula: α, β are the weights of the cross entropy loss function and the Dice loss function respectively; M is the brain region segmentation category; N is the total number of pixels on each slice; is the predicted value of the i-th pixel of the output image on the j-th brain region segmentation category; It is the true value of the i-th pixel of the MRI slice image in the j-th brain region segmentation category.

6. The device according to claim 5, characterized in that The Unet module is a 7-layer Unet model; The 7-layer Unet model includes 14 convolution modules. In the first 6 convolution modules, a downsampling module is set after each convolution module. From the 8th to the 13th convolution module, an upsampling module is set before each convolution module. The downsampling module uses maximum pooling to compress the input image; The upsampling module uses inverse convolution to amplify the input image; In the first 13 convolutional modules, each convolutional module has two convolutional layers for feature extraction, and a normalization layer and an activation layer are added to each convolutional layer in sequence; The 14th convolutional module has only one convolutional layer, which is used to output various brain region segmentation results; The normalization method adopted by the normalization layer is instance normalization, and the function of the activation layer selects the Leaky ReLU activation function.

7. The device according to claim 5, characterized in that There are 43 brain region labels, including 42 brain region segmentation labels and 1 background.

8. The device according to claim 5, characterized in that The loss function is optimized using the Adam optimizer.

Images