A Brain Tissue Image Segmentation Method Incorporating Prior Information and Feature Fusion

Abstract

The present invention relates to a method for brain tissue image segmentation by introducing prior information and feature fusion, belonging to the fields of digital image processing and pattern recognition. The method comprises the following steps: preprocessing the brain tissue image and dividing the data into a training set and a validation set; extracting features of data blocks through a dense connection module and a bottleneck module; fusing features of different hierarchical encoding stages at the skip connection part of each backbone network and transmitting them to the decoding layer; constructing the decoding layer using a residual connection module and upsampling; using three backbone networks to segment three brain tissues respectively, and introducing the three segmentation results as prior information into the network. The present invention introduces feature fusion to cope with feature extraction of complex structures, and introduces prior information to enable the network to pay more attention to error-prone areas, thereby improving the segmentation accuracy of the network.

Description

Technical Field

[0001] The present invention belongs to the field of digital image processing and pattern recognition, and relates to a brain tissue image segmentation method introducing prior information and feature fusion. Background Art

[0002] The brain is one of the most important organs of the human body and is crucial for human physical health. Research shows that the development status of the brain is closely related to human physical health. By studying the development status of the brain, potential risks in the brain can be evaluated and predicted in advance. With the rapid development of medical imaging technology, medical imaging techniques are widely used in clinical auxiliary diagnosis. And accurate segmentation of brain tissue is a key step in assisting doctors in quantitative and qualitative analysis. Manual segmentation of brain tissue not only requires rich medical knowledge but also has low segmentation efficiency and is easily affected by doctors' subjective factors. Therefore, a fast and accurate automated segmentation algorithm becomes very crucial.

[0003] The human brain has a highly complex structure, containing various different types of tissues, such as gray matter, white matter, cerebrospinal fluid, etc., and their shapes, sizes, and positions vary significantly. In addition, there are complex interactions between these tissues, making the structure of the entire brain more complex. Affected by the sampling sequence or radiofrequency coil, even the gray levels of the same anatomical structure in the same image often have uneven distributions. All these will affect the segmentation of brain tissue.

[0004] Due to its excellent performance, U-Net is widely used in the field of medical image segmentation. However, since U-Net cannot extract sufficient spatial information, researchers have added attention mechanisms, dense connections, etc. to improve the network's feature extraction ability on this network structure. Due to the relatively complex structure of the brain tissue, simply changing the network structure has limited improvement in performance. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a brain tissue image segmentation method based on prior information and feature fusion, which pays attention to the channel and spatial information of the image through a feature fusion module with hybrid attention to provide more effective and accurate image features; and uses the prior information learned by the model to further guide the segmentation process of the model, thereby improving the segmentation accuracy of the model.

[0006] To achieve the above purpose, the present invention provides the following technical solutions:

[0007] A brain tissue image segmentation method introducing prior information and feature fusion, characterized in that: the method includes the following steps:

[0008] S1. Obtain and preprocess the brain tissue image, and divide the preprocessed image into a training set and a validation set;

[0009] S2. Use the dense connection module and the bottleneck module to extract image features to construct an encoding module, and extract complex medical image features;

[0010] S3. Input the image features obtained in step S2 into the feature fusion module in the skip connection;

[0011] S4. Combine the features in the feature fusion stage with the upsampled features, and use the residual connection module to build a decoding module;

[0012] S5. Take the segmentation results of the three backbone networks as prior information and introduce them into the network to obtain the segmentation results of the model.

[0013] Further, step S1 includes the following steps:

[0014] S11. Crop the obtained three-dimensional brain tissue image, remove the redundant regions with pixel gray value of 0, and retain the smallest cube containing the complete brain tissue region;

[0015] S12. Perform Z-Score normalization on the cropped image;

[0016] S13. Cut the processed brain tissue image data into several image blocks of size 64×64×64, and randomly select one block as the input of the U-Net network model.

[0017] Further, step S2 includes the following steps:

[0018] S21. Use the dense connection module and the bottleneck module to extract features from the brain tissue image to obtain abstract semantic features of 64×64×64;

[0019] S22. Use max pooling and downsampling to compress the feature maps in step S21;

[0020] S23. Repeat steps S21 and S22, and finally obtain a feature map of 8×8×8.

[0021] Further, step S3 includes the following steps:

[0022] S31. Pass the feature maps of the previous encoding layer to the skip connection stage of the current encoding layer through downsampling operations;

[0023] S32. Use the feature fusion module to fuse the feature maps of the current encoding layer and the feature maps obtained through step S31, and pass the fused feature maps to the decoding layer.

[0024] Further, step S4 includes the following steps:

[0025] S41. Add the feature map obtained in step S3 to the feature map after the upsampling operation. The size of the image is doubled after each upsampling operation.

[0026] S42. Transmit the feature map obtained in step S41 to the residual connection module.

[0027] S43. Repeat steps S41 and S42 until the feature map is restored to the same size as the input image.

[0028] Further, step S5 includes the following steps:

[0029] S51. Use three backbone networks to separately segment different tissues in the brain image.

[0030] S52. Stitch the three segmentation results obtained in step S51 with the original input image and use them as the input of a new backbone network. After steps S2 - S4, the final segmentation result of the U - Net model is obtained.

[0031] The beneficial effects of the present invention are as follows: By introducing feature fusion, the present invention can handle feature extraction of complex structures. By introducing prior information, the network can pay more attention to error - prone areas, thereby improving the segmentation accuracy of the network model.

[0032] Other advantages, objectives, and features of the present invention will, to some extent, be elaborated in the subsequent description. And to some extent, they will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0034] Figure 1 is a schematic flowchart of the method for segmenting brain tissue images of the present invention;

[0035] Figure 2 is a schematic diagram of the network of the image segmentation model;

[0036] Figure 3 is a schematic diagram of the backbone network;

[0037] Figure 4 is a schematic diagram of the structure of the feature fusion module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following specific examples illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present invention. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0039] Among them, the attached drawings are only for illustrative purposes, showing only schematic diagrams rather than physical diagrams, and should not be construed as a limitation on the present invention; in order to better illustrate the embodiments of the present invention, some components in the attached drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the attached drawings may be omitted.

[0040] In the attached drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or position relationship, they are based on the orientation or position relationship shown in the attached drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the attached drawings are only for illustrative purposes and should not be construed as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0041] As Figure 1 shown, it is a brain tissue image segmentation method introducing prior information and feature fusion, and its specific steps are as follows:

[0042] S1. Obtain and preprocess the three-dimensional brain image, and randomly sample a pixel block of size 64×64×64 as the input. The specific steps are as follows:

[0043] S11. Crop the entire three-dimensional medical image, remove the redundant areas with pixel gray value of 0, and retain the smallest cube containing the complete brain tissue area;

[0044] S12. Perform Z-Score normalization on the cropped image to make the image follow a normal distribution with a mean of 0 and a standard deviation of 1;

[0045] S13. Cut the processed three-dimensional medical image data into image blocks of size 64×64×64, and randomly select one block as the input of the U-Net model. Among them, the image segmentation model in the present invention is as Figure 2as shown

[0046] S2. Input the image into the feature encoding module to extract image features with texture and semantic information, specifically as follows:

[0047] S21. Use a dense connection module and a bottleneck module to extract features from the three-dimensional image;

[0048] S22. Compress the feature map obtained in step S21 using max pooling and downsampling;

[0049] S23. Repeat step S21 and step S22 to finally obtain a feature map of 8×8×8 in the encoding layer.

[0050] S3. Send the feature map obtained in step S2 into the feature fusion module in the skip connection to better extract features, specifically as follows:

[0051] S31. Transmit the feature map of the previous encoding layer to the skip connection stage of the current encoding layer after downsampling;

[0052] S32. Use a feature fusion module, as Figure 4 shown, to fuse the feature map of the current encoding layer and the feature map obtained in step S31, and transmit the fused feature map to the decoding layer.

[0053] S4. Transmit the features in the feature fusion stage and the upsampled features to the residual connection module, specifically as follows:

[0054] S41. Add the feature map obtained in step S3 and the feature map after upsampling. The image size is doubled after each upsampling;

[0055] S42. Transmit the feature map obtained in step S41 to the residual connection module;

[0056] S43. Repeat step S41 and step S42 to finally restore the feature map to the same size as the input image.

[0057] S5. Use the segmentation results of the three backbone networks as prior information and introduce them into the network to obtain the segmentation result of the model, specifically as follows:

[0058] S51. Use three backbone networks to separately segment different tissues of the brain image, as Figure 3 shown;

[0059] S52. Concatenate the 3 segmentation results obtained in step S51 with the original input as the input of the new backbone network, and obtain the final segmentation result of the model through steps S2, S3, and S4.

[0060] To verify the effectiveness of the present invention, the following experiments were conducted:

[0061] The brain tissue image segmentation method based on prior information and feature fusion is tested on the iSeg2017 dataset. Ten samples in the iSeg2017 dataset are used for experiments. The segmentation target of this dataset is to segment infant brain MR images into cerebrospinal fluid, gray matter, and white matter. The network is trained using the ten-fold cross-validation method. Nine brain samples are randomly selected to train the neural network, and the remaining one sample is used for testing. At the same time, Method 1 of U-Net, Method 2 which uses a dense connection module in the encoding stage and a residual connection module in the decoding stage based on Method 1, Method 3 which uses a feature fusion module in the skip connection based on Method 2, and Method 4 which adds prior information based on Method 3 are compared. The Dice coefficient is used as the evaluation index. The larger the DICE value, the better the segmentation result. The DICE formula is as follows:

[0062]

[0063] Where T represents the segmentation result of the network, and R represents the gold standard given by the dataset.

[0064] Table 1 shows the test results on the iSeg2017 dataset. It can be seen that the DICE values of Method 1 to Method 4 are getting larger and larger, indicating that the segmentation performance of the model is getting better and better.

[0065] Table 1 Test Results on the iSeg2017 Dataset

[0066] CSF WM GM AVG Method 1 93.69 90.37 90.32 91.46 Method 2 94.48 91.04 91.42 92.31 Method 3 94.77 91.49 91.76 92.67 Method 4 95.03 91.86 92.28 93.06

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A brain tissue image segmentation method that introduces prior information and feature fusion, characterized in that: The method includes the following steps: S1. Obtain and preprocess a brain tissue image, and divide the preprocessed image into a training set and a validation set; S2. Use a dense connection module and a bottleneck module to extract image features to construct an encoding module, and extract complex medical image features, mainly including: S21. Use a dense connection module and a bottleneck module to extract features from the brain tissue image to obtain abstract semantic features of 64×64×64; S22. Use max pooling and downsampling to compress the feature map in step S21; S23. Repeat steps S21 and S22 until a feature map of 8×8×8 is finally obtained; S3. Input the image features obtained in step S2 into the feature fusion module in the skip connection, mainly including: S31. Pass the feature map of the previous encoding layer through a downsampling operation to the skip connection stage of the current encoding layer; S32. Use the feature fusion module to fuse the feature map of the current encoding layer and the feature map obtained through step S31, and pass the fused feature map to the decoding layer; S4. Combine the features in the feature fusion stage with the upsampled features, and use a residual connection module to build a decoding module, mainly including: S41. Add the feature map obtained in step S3 to the feature map after the upsampling operation. The image size is doubled after each upsampling; S42. Transmit the feature map obtained in step S41 to the residual connection module; S43. Repeat steps S41 and S42 until the feature map is restored to the same size as the input image; S5. Use the segmentation results of the three backbone networks as prior information and introduce them into the network to obtain the segmentation result of the model, mainly including: S51. Use the three backbone networks to separately segment different tissues in the brain image; S52. Stitch the three segmentation results obtained in step S51 with the original input image, and use it as the input of the new backbone network. After steps S2 to S4, the final segmentation result of the U-Net model is obtained.

2. The brain tissue image segmentation method according to claim 1, wherein: Step S1 includes the following steps: S11. Crop the obtained three-dimensional brain tissue image, remove the redundant area with a pixel gray value of 0, and retain the smallest cube containing the complete brain tissue area; S12. Perform Z-Score normalization on the cropped image; S13. Cut the processed brain tissue image data into several image patches of 64×64×64 in size, and randomly select one patch as the input of the U-Net network model.

Images