Brain tumor image region segmentation method and device, neural network and electronic equipment

Abstract

The application relates to a brain tumor image region segmentation method and device, a neural network and electronic equipment, and comprises the following steps: acquiring a brain MRI image, forming a data set, and pre-processing the data set; an improved residual attention block is constructed, and a multilayer perceptron therein is replaced; an improved network model is constructed, the number of the improved residual attention block is adjusted, and a replacement method is replaced by introducing the improved residual attention block; the trained network model is input into a test set for testing, and the network effect is verified; the application has the beneficial effects that the application is based on a convolutional neural network, proposes a brain tumor segmentation method based on an improved residual attention mechanism of UNet, that is, a double-branch model structure, compensates for the problem that UNet cannot utilize global information of an image, and can effectively improve model segmentation quality.

Description

Technical Field

[0001] This invention relates to the field of deep neural network technology, and more specifically, to brain tumor image region segmentation methods, devices, neural networks, and electronic devices. Background Technology

[0002] Brain tumors are tumors that form within the brain or under the meninges. Currently, humans are affected by more than 100 types of brain tumors. Among them, gliomas are the most common malignant brain tumors, originating from glial cells, and usually cause some degree of invasion and compression of surrounding brain tissue. Therefore, the diagnosis and surgical prediction of brain tumors have become increasingly important. With the rapid development of artificial intelligence, AI-based tumor prediction and diagnosis technologies are becoming increasingly sophisticated. Using techniques such as voxel analysis, accurate and precise segmentation of brain tumors can be achieved, which is helpful for preoperative planning. In addition, lifespan can also be predicted from the segmented tumor. Precise segmentation of tumors from brain tumor images is currently at the forefront of research.

[0003] Magnetic resonance imaging (MRI) plays a crucial role in the diagnosis, treatment planning, and monitoring of brain tumors. MRI provides high-resolution images, helping physicians detect and differentiate brain tumor types, locate tumor positions, assess tumor size, morphology, invasiveness, and blood supply, and determine whether the tumor has affected surrounding structures. MRI scans can generate various image sequences, including T1-weighted (T1), T1-contrast-enhanced (T1-ce), T2-weighted (T2), and T2 fluid attenuation inversion recovery (FLAIR). These image sequences provide different anatomical and physiological information. T1-weighted images can show the anatomical location and size of the tumor, T2-weighted images can show features such as tumor invasiveness and cystic changes, FLAIR images can show surrounding edema and brain tissue destruction, and contrast-enhanced sequences can show the boundary between the tumor and surrounding normal brain tissue and the degree of vascularization.

[0004] With the development of image segmentation techniques based on deep learning methods, automated medical image segmentation can achieve accurate segmentation of brain tumor images. In recent years, deep learning-based brain tumor segmentation techniques have achieved state-of-the-art performance on various benchmarks, mainly because convolutional neural networks (CNNs) have a powerful feature extraction capability. However, the convolution operation in CNNs extracts features based on local neighborhood pixel information without directly considering the relationships between global pixels. This method may fail to capture some global features, such as long-distance dependencies or global semantic information, thus limiting the performance of CNNs. Yang et al. added dilated convolutional blocks to the network to increase the receptive field. Other efforts have also improved the network's ability to extract and integrate semantic information by increasing the receptive field, but the receptive field of these methods is still limited to local regions. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a brain tumor image region segmentation method, device, neural network, and electronic device.

[0006] Firstly, a brain tumor image region segmentation method is provided, including:

[0007] S1. Acquire brain MRI images, form a dataset, and preprocess the dataset;

[0008] S2. Construct an improved residual attention block and replace the multilayer perceptron within it;

[0009] S3. Construct an improved network model by introducing the improved residual attention block and adjusting the number and replacement method of the improved residual attention block;

[0010] S4. Input the trained network model into the test set for testing to verify the network performance.

[0011] Preferably, in S1, the brain MRI images have multiple modalities, and the images of each modality are standardized using the Z-score method, as follows:

[0012] X*=(X-μ) / σ

[0013] Where μ is the mean of all sample data and σ is the standard deviation of all sample data.

[0014] Preferably, in S1, the preprocessing includes: mirror flipping, rotation, scaling, translation, elastic deformation, and image cropping.

[0015] Preferably, S2 includes:

[0016] S201. Replace the multilayer perceptron layer in the original attention block and add residual connections on the basis of the multilayer perceptron.

[0017] S202, then add the GELU activation function, a random deactivation layer, and a linear layer.

[0018] Preferably, in S3, the improved network model includes an encoder part and a decoder part; the encoder part consists of three layers of encoding blocks, namely 3D convolutional layers, batch normalization layers and LeakyReLu layers; the decoder part includes decoding blocks, which are used to decode the feature maps to obtain the brain tumor region.

[0019] Preferably, in S3, the last three layers of the encoder section use dual-branch improved residual attention blocks, with the number of improved residual attention blocks being (2, 4, 2). Among them, the odd-numbered residual attention blocks consist of a linear normalization layer, W-MSA, batch normalization layer, and residual multilayer sensor machine layer; the even-numbered residual attention blocks consist of a linear normalization layer, SW-MSA, batch normalization layer, and residual multilayer sensor machine layer.

[0020] As a preferred method, in S4, the preprocessed image is input into the network to update the network parameter weights and obtain the optimal network segmentation result; the segmentation result is then processed by the sigmoid function to convert the segmentation result into 0 and 1, which are then concatenated and restored to a single channel according to the three-channel definition to obtain the segmentation result image.

[0021] In a second aspect, a brain tumor image region segmentation apparatus is provided for performing any of the brain tumor image region segmentation methods described in the first aspect, including:

[0022] An acquisition module is used to acquire brain MRI images, form a dataset, and preprocess the dataset;

[0023] The first building block is used to build an improved residual attention block, replacing the multilayer perceptron within it;

[0024] The second construction module is used to construct the improved network model, introduce the improved residual attention block, and adjust the number and replacement method of the improved residual attention block;

[0025] The testing module is used to input the trained network model into the test set for testing and to verify the network performance.

[0026] Thirdly, a computer storage medium is provided, wherein a computer program is stored therein; when the computer program is run on a computer, the computer performs the brain tumor image region segmentation method described in any of the first aspects.

[0027] Fourthly, an electronic device is provided, comprising: a processor and a memory, wherein the memory stores a program; the processor is configured to invoke the program stored in the memory to cause the electronic device to perform the brain tumor image region segmentation method as described in any of the first aspects.

[0028] The beneficial effects of this invention are:

[0029] 1. This invention proposes a brain tumor segmentation method based on a residual attention mechanism improved from UNet, namely a two-branch model structure, which overcomes the problem that UNet cannot utilize the global information of the image and can effectively improve the segmentation quality of the model.

[0030] 2. This invention improves the attention block by replacing the multi-layer perceptron layer and introducing residual connections in the multi-layer perceptron layer, which can effectively solve the problem of gradient vanishing caused by deepening the network. Attached Figure Description

[0031] Figure 1 Flowchart of the brain tumor image region segmentation method provided by the present invention;

[0032] Figure 2 A schematic diagram of the network model for the brain tumor image region segmentation method provided by the present invention;

[0033] Figure 3 This is a schematic diagram of the residual attention mechanism model provided by the present invention;

[0034] Figure 4 A schematic diagram of the residual multilayer perceptron layer structure in the improved residual attention block provided by the present invention;

[0035] Figure 5 This is a schematic diagram of the brain tumor image region segmentation device provided by the present invention. Detailed Implementation

[0036] The present invention will be further described below with reference to embodiments. The description of the embodiments below is only for the purpose of helping to understand the present invention. It should be noted that those skilled in the art can make several modifications to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0037] Example 1:

[0038] This application provides a brain tumor segmentation method based on an improved residual attention mechanism of UNet. By introducing an attention mechanism for optimization, the problem that the UNet network does not make good use of local and total information is solved. In the attention block, a residual multilayer perceptron layer is used instead of a multilayer perceptron layer. Residual connections are added on the basis of the multilayer perceptron layer, which can make the network converge faster, thereby improving the segmentation ability of the model.

[0039] Specifically, such as Figure 1 As shown, the above method includes:

[0040] S1. Acquire brain MRI images, form a dataset, and preprocess the dataset.

[0041] In S1, the brain MRI images have four modalities (t1, t2, flair, t1ce). The images for each modality are standardized using the Z-score method, and are represented as follows:

[0042] X*=(X-μ) / σ

[0043] Where μ is the mean of all sample data, σ is the standard deviation of all sample data, and X is the input data of the original image.

[0044] In addition, to improve the model's generalization ability and robustness, methods such as mirror flipping, rotation, scaling, translation, and elastic deformation are used.

[0045] In MRI images of brain tumors, the brain region appears as gray, while the black area represents the background. Background information constitutes a large portion of the overall image, and it offers no benefit for segmentation. Therefore, background removal is necessary. The original image size (160, 240, 240) was cropped to (128, 128, 128).

[0046] For example, four-modality brain MRI images are first read from the brain tumor segmentation dataset BraTs2020. After background cropping and image enhancement, the input I∈R^(C×H×W×D) of the encoding path is obtained, where C represents the number of modalities or channels (4 in practice), and H, W, and D represent the length, width, and depth (128 in practice), respectively. I is then processed by feature extraction and downsampling through each encoding module in the encoding path to obtain the output I^'. The number of encoding modules in the encoding path, the structure of the encoding modules, and the structure of the encoding blocks all reference classic network structures in the field of brain tumor segmentation, such as 3D U-Net.

[0047] S2. Construct an improved residual attention block and replace the multilayer perceptron within it.

[0048] Specifically, S2 includes:

[0049] S201. Replace the multilayer perceptron layer in the original attention block and add a residual connection on the basis of the multilayer perceptron. This will add an unprocessed input to the output of the multilayer perceptron.

[0050] S202, such as Figure 4 As shown, the GELU activation function, a dropout layer, and a linear layer are then added.

[0051] An alternative multilayer perceptron layer is represented as follows:

[0052]

[0053]

[0054] Where GELU represents the GELU nonlinear layer, LN represents the linear layer, and α represents the learnable weights of the linear layer. Represents the output of the l-th layer of the neural network. This represents the final output.

[0055] S3. Construct an improved network model by introducing the improved residual attention block and adjusting the number and replacement method of the improved residual attention block.

[0056] This improved network model, based on UNet, replaces the last three layers of the encoder with residual attention blocks. Specifically, the number of residual attention blocks in the encoder is (2, 4, 2), and they are set to a two-branch form.

[0057] S4. Input the trained network model into the test set for testing to verify the network performance.

[0058] Example 2:

[0059] Based on Example 1, Example 2 of this application provides a more specific method for brain tumor image region segmentation, including:

[0060] S1. Acquire brain MRI images, form a dataset, and preprocess the dataset.

[0061] S2. Construct an improved residual attention block and replace the multilayer perceptron within it.

[0062] S3. Construct an improved network model by introducing the improved residual attention block and adjusting the number and replacement method of the improved residual attention block.

[0063] In S3, such as Figure 2 As shown, the improved network model includes an encoder and a decoder. The first three layers of the encoder consist of 3D convolutional layers, batch normalization layers, and LeakyReLU layers. Specifically, the first layer of the encoder consists of 3D convolutional layers with a kernel size of 3×3×3, a stride of 1×1×1, and padding of 1×1×1, along with batch normalization layers and LeakyReLU layers. The first encoding block of the second to third layers consists of 3D convolutional layers with a kernel size of 3×3×3, a stride of 2×2×2, and padding of 1×1×1, along with batch normalization layers and LeakyReLU layers, serving as a downsampling layer.

[0064] Specifically, the last three layers of the encoder section use improved residual attention blocks with two branches, and the number of improved residual attention blocks is (2, 4, 2); where, as Figure 3 As shown, the single-number residual attention block consists of a linear normalization layer, a W-MSA layer, and a residual multilayer perceptron layer; the double-number residual attention block consists of a linear normalization layer, a W-MSA layer, and a residual multilayer perceptron layer.

[0065] The complete residual attention block is represented as:

[0066]

[0067]

[0068]

[0069]

[0070] W-MSA and SW-MSA are multi-head attention blocks with regular and shifted window configurations, respectively. and These are the outputs of W-MSA and SW-MSA, respectively. ResMLP is the residual multilayer perceptron mentioned above.

[0071] To fully and effectively utilize the computational shift window mechanism, this application employs three-dimensional cyclic shifting, the process of which is represented as follows:

[0072]

[0073] Where Q, K, and V represent the query, key, and value, respectively; d represents the size of the query and the key.

[0074] The decoder section includes decoding blocks used to decode the feature maps to obtain the brain tumor regions. Specifically, the upsampling layer is implemented using a 3D deconvolutional layer with a kernel size of 3×3×3, a stride of 2×2×2, and padding of 1×1×1. The channel adjustment layer includes a 3D convolutional layer with a kernel size of 3×3×3, a stride of 1×1×1, and padding of 1×1×1. The number of output channels of the 3D convolutional layer is consistent with the number of glioma sub-regions. The number of decoding modules in the decoding path, the structure of the decoding modules, and the structure of the decoding blocks all reference classic network structures in the field of brain tumor segmentation, such as 3D U-Net.

[0075] S4. Input the trained network model into the test set for testing to verify the network performance.

[0076] In S4, the preprocessed image is input into the network to update the network parameter weights and obtain the optimal network segmentation result. The segmentation result is then processed by the sigmoid function to convert the segmentation result into 0s and 1s, which are then concatenated and restored to a single channel according to the three-channel definition to obtain the segmentation result image.

[0077] It should be noted that the parts in this embodiment that are the same as or similar to those in Embodiment 1 can be referred to each other, and will not be repeated in this application.

[0078] Example 3:

[0079] Based on Embodiment 1, Embodiment 3 of this application provides a brain tumor image region segmentation device, such as... Figure 5 As shown, it includes:

[0080] An acquisition module is used to acquire brain MRI images, form a dataset, and preprocess the dataset;

[0081] The first building block is used to build an improved residual attention block, replacing the multilayer perceptron within it;

[0082] The second construction module is used to construct the improved network model, introduce the improved residual attention block, and adjust the number and replacement method of the improved residual attention block;

[0083] The testing module is used to input the trained network model into the test set for testing and to verify the network performance.

[0084] Specifically, the device provided in this embodiment is the same as the device corresponding to the method provided in Embodiment 1. Therefore, the parts in this embodiment that are the same as or similar to those in Embodiment 1 can be referred to each other, and will not be repeated in this application.

Claims

1. A brain tumor image region segmentation method, characterized in that, include: S1. Acquire brain MRI images, form a dataset, and preprocess the dataset; S2. Construct an improved residual attention block and replace the multilayer perceptron within it; S2 includes: S201. Replace the multilayer perceptron layer in the original attention block and add residual connections on the basis of the multilayer perceptron. S202, then add the GELU activation function, a random deactivation layer, and a linear layer; S3. Construct an improved network model by introducing the improved residual attention block and adjusting the number and replacement method of the improved residual attention block; In S3, the improved network model includes an encoder and a decoder. The first three layers of the encoder consist of 3D convolutional layers, batch normalization layers, and LeakyReLu layers. The decoder includes decoding blocks used to decode the feature maps to obtain the brain tumor region. In S3, the last three layers of the encoder use dual-branch improved residual attention blocks, with a number of (2, 4, 2). The odd-numbered residual attention blocks consist of a linear normalization layer, W-MSA, a batch normalization layer, and a residual multilayer perceptron layer. The even-numbered residual attention blocks consist of a linear normalization layer, W-MSA, a batch normalization layer, and a residual multilayer perceptron layer. S4. Input the trained network model into the test set for testing to verify the network performance.

2. The brain tumor image region segmentation method according to claim 1, characterized in that, In S1, the brain MRI images have multiple modalities. The Z-score method is used to standardize the images of each modality, as follows: X*=(X-μ) / σ Where μ is the mean of all sample data and σ is the standard deviation of all sample data.

3. The brain tumor image region segmentation method according to claim 2, characterized in that, In S1, the preprocessing includes: mirror flipping, rotation, scaling, translation, elastic deformation, and image cropping.

4. The brain tumor image region segmentation method according to claim 3, characterized in that, In S4, the preprocessed image is input into the network to update the network parameter weights and obtain the optimal network segmentation result. The segmentation result is then processed by the sigmoid function to convert the segmentation result into 0s and 1s, which are then concatenated and restored to a single channel according to the three-channel definition to obtain the segmentation result image.

5. A brain tumor image region segmentation device, characterized in that, A method for performing brain tumor image region segmentation according to any one of claims 1 to 4 includes: An acquisition module is used to acquire brain MRI images, form a dataset, and preprocess the dataset; The first building block is used to build an improved residual attention block, replacing the multilayer perceptron within it; The second construction module is used to construct the improved network model, introduce the improved residual attention block, and adjust the number and replacement method of the improved residual attention block; The testing module is used to input the trained network model into the test set for testing and to verify the network performance.

6. A computer storage medium, characterized in that, The computer storage medium stores a computer program; when the computer program is run on the computer, it causes the computer to execute the brain tumor image region segmentation method according to any one of claims 1 to 4.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores a program; the processor is configured to invoke the program stored in the memory to cause the electronic device to perform the brain tumor image region segmentation method as described in any one of claims 1 to 4.

Images