Brain tumor segmentation method and device based on improved 3DU-Net, and electronic equipment

Abstract

The invention relates to a brain tumor segmentation method and device based on improved 3D U-Net, and electronic equipment, and the method comprises the steps: obtaining multi-modal brain tumor data, carrying out the preprocessing of the multi-modal brain tumor data, obtaining to-be-processed brain tumor data, constructing a brain tumor segmentation model of the multi-scale feature mining of the improved 3D U-Net, and obtaining the to-be-processed brain tumor data. And training the brain tumor segmentation model by using the to-be-processed brain tumor data, storing an optimal weight model in the training process, and performing model prediction and performance index calculation by using the optimal weight model. According to the method, feature information of different scales can be more effectively captured and integrated by introducing a multi-scale feature mining module, so that the accuracy and robustness of segmentation are improved. In addition, the method utilizes a deep learning technology to realize full-automatic segmentation of the brain tumor. Compared with a traditional manual segmentation method, the method has the advantages that the segmentation efficiency can be remarkably improved, the workload of doctors is relieved, and segmentation errors caused by human factors are reduced.

Description

Technical Field

[0001] The present application relates to the technical field of medical image analysis, and in particular to a brain tumor segmentation method, device and electronic device based on an improved 3D U-Net. Background Art

[0002] In the field of medical image processing, brain tumor segmentation is a critical and complex task. Correct segmentation of brain tumors is crucial for clinical diagnosis and treatment planning, because it is directly related to key information such as tumor location, size, shape, and boundaries with other tissues. General brain tumor segmentation methods are generally divided into traditional methods and deep learning-based methods. Traditional manual segmentation methods rely on the doctor's experience and subjective judgment, and have problems such as low efficiency, strong subjectivity, and poor repeatability. Traditional automatic segmentation methods, such as threshold-based and region growing-based algorithms, can achieve a certain degree of automation, but often have difficulty in handling the complexity and variability of brain tumor images. Deep learning-based methods can automatically learn feature representations in images and train and optimize through a large amount of data to achieve high-precision segmentation. Although deep learning technology has made significant progress in brain tumor segmentation, there are still some challenges, such as single grayscale information, high grayscale similarity between structural edges and normal tissues, and variable tumor size and shape. Therefore, developing an automatic, accurate, and efficient brain tumor segmentation system has important clinical value and practical significance. Summary of the invention

[0003] In view of this, the present application proposes a brain tumor segmentation method based on an improved 3D U-Net to solve the problems reflected in the above-mentioned background technology.

[0004] According to one aspect of the present application, a brain tumor segmentation method based on an improved 3D U-Net is provided, comprising the following steps:

[0005] Acquiring multimodal brain tumor data, and preprocessing the multimodal brain tumor data to obtain brain tumor data to be processed;

[0006] Construct a brain tumor segmentation model based on multi-scale feature mining of improved 3D U-Net;

[0007] Using the brain tumor data to be processed to train the brain tumor segmentation model, and saving the optimal weight model during the training process;

[0008] The optimal weight model is used to perform model prediction and calculate performance indicators.

[0009] As an optional implementation scheme of the present application, optionally, obtaining multimodal brain tumor data, and preprocessing the multimodal brain tumor data to obtain brain tumor data to be processed, includes:

[0010] The multimodal brain tumor data is the public glioma dataset BraTS2021 provided by the International Association for Medical Image Computing and Computer-Assisted Intervention;

[0011] The training set in the multimodal brain tumor data is preprocessed by using random flipping, random cropping, denoising and other methods to obtain the brain tumor data to be processed.

[0012] As an optional implementation scheme of the present application, optionally, a brain tumor segmentation model for multi-scale feature mining of an improved 3D U-Net is constructed, including three parts: an encoder, a multi-scale feature mining module and a decoder.

[0013] As an optional implementation scheme of the present application, optionally, the brain tumor segmentation model is trained using the brain tumor data to be processed, and the optimal weight model in the training process is saved, including:

[0014] Set hyperparameters;

[0015] Training the constructed brain tumor segmentation model according to the brain tumor data to be processed;

[0016] Save the optimal weight model during training.

[0017] As an optional implementation scheme of the present application, optionally, using the optimal weight model to perform model prediction and calculate performance indicators includes:

[0018] Use the optimal weight model to predict the validation set, obtain and save the prediction results;

[0019] The performance index is calculated based on the predicted results and the true labels.

[0020] As an optional implementation scheme of the present application, optionally, the constructed brain tumor segmentation model includes three parts: an encoder, a multi-scale feature mining module and a decoder, including:

[0021] The multi-scale feature mining module also includes a key feature mining module FMM (Feature mining module).

[0022] As an optional implementation scheme of the present application, optionally, it is characterized in that, according to the brain tumor data to be processed, the constructed brain tumor segmentation model is trained, including:

[0023] The mixed loss function BCEDice Loss is used to calculate the error value between the predicted value output by the brain tumor segmentation model and the true label value, so as to improve the effect of model training.

[0024] According to the second aspect of the present application, a brain tumor segmentation device based on an improved 3D U-Net is provided, comprising the following modules:

[0025] A data acquisition and data preprocessing module, used for acquiring multimodal brain tumor data and preprocessing the multimodal brain tumor data;

[0026] A model building module is used to build a brain tumor segmentation model with improved multi-scale feature mining;

[0027] A training model module, used to train the constructed brain tumor segmentation model to obtain an optimal weight model;

[0028] The model prediction and calculation module is used to use the optimal weight model to predict the validation set and obtain the prediction results and calculated performance indicators.

[0029] As an optional implementation scheme of the present application, optionally, building a model module includes:

[0030] an encoder module for extracting features from the multimodal brain tumor data;

[0031] A multi-scale mining module is used to obtain contextual information at different scales, which helps to improve the accuracy of brain tumor segmentation;

[0032] The decoder module is used to gradually restore the size of the feature map to its original size to generate the final brain tumor segmentation result.

[0033] According to the third aspect of the present application, an electronic device is proposed, comprising:

[0034] processor;

[0035] a memory for storing processor-executable instructions;

[0036] Wherein, the processor is configured to implement a brain tumor segmentation method based on an improved 3D U-Net as described in any one of claims 1 to 8 when executing the executable instructions.

[0037] Beneficial effects of this application:

[0038] The innovation and advancement of this application are reflected in the following aspects: traditional brain tumor segmentation methods often fail to achieve ideal segmentation effects when dealing with brain tumors with diverse shapes, uneven distribution of positions and sizes, and complex boundaries. The improved 3D U-Net can more effectively capture and integrate feature information of different scales by introducing multi-scale feature mining, thereby improving the accuracy and robustness of segmentation. In addition, the system uses deep learning technology to achieve fully automated segmentation of brain tumors. Compared with traditional manual segmentation methods, this method can significantly improve segmentation efficiency, reduce the workload of doctors, and reduce segmentation errors caused by human factors.

[0039] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A flowchart of a brain tumor segmentation method based on an improved 3D U-Net according to an embodiment of the present application is shown;

[0042] Figure 2 A block diagram of a brain tumor segmentation device based on an improved 3D U-Net according to an embodiment of the present application is shown;

[0043] Figure 3 A structural diagram showing a brain tumor segmentation model constructed in a brain tumor segmentation method based on an improved 3D U-Net in an embodiment of the present application;

[0044] Figure 4 (a) shows the structure of DoubleConv in the brain tumor segmentation method based on improved 3D U-Net in an embodiment of the present application;

[0045] Figure 4 (b) shows a structural diagram of the ASPP module in the brain tumor segmentation method based on the improved 3D U-Net according to an embodiment of the present application;

[0046] Figure 4 (c) shows the structural diagram of the FMM module in the brain tumor segmentation method based on the improved 3D U-Net in an embodiment of the present application. DETAILED DESCRIPTION

[0047] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0048] Among them, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application or simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0050] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0051] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0052] The present application scheme is mainly used in the clinical diagnosis, surgical planning and treatment evaluation of brain tumor patients. The present application provides a brain tumor segmentation system for clinical use with improved multi-scale feature mining of 3D U-Net. 3D U-Net is a medical image segmentation model based on deep learning. The model realizes efficient processing of 3D images through the symmetrical structure of the encoder and the decoder and the jump connection method. However, when processing the brain tumor segmentation task, due to the particularity of the tumor in the MRI image, the traditional 3D U-Net model is often difficult to achieve the ideal segmentation effect. The present application introduces an improved 3D U-Net model including a multi-scale feature mining module. The multi-scale feature mining module can more comprehensively capture the features of the brain tumor in the image by extracting and fusing information on feature maps of different scales, improve the accuracy and efficiency of brain tumor image segmentation, and enable doctors to obtain information such as the location, morphology and size of the brain tumor more quickly and accurately, providing strong support for the formulation of personalized treatment plans. The multi-scale feature mining module also enables the system to better pay attention to the feature information of the brain tumor part by adding an attention mechanism. When the system is used to segment brain tumors of different forms, a better segmentation effect can be achieved.

[0053] Example 1

[0054] Figure 1 FIG. 1 is a flowchart of a brain tumor segmentation method based on an improved 3D U-Net according to Embodiment 1 of the present application. Figure 1 As shown, the flow chart includes:

[0055] S100, acquiring multimodal brain tumor data, and preprocessing the multimodal brain tumor data to obtain brain tumor data to be processed.

[0056] In this embodiment, the multimodal brain tumor data is the public glioma dataset BraTS2021 provided by the International Association for Medical Image Computing and Computer-Assisted Intervention. The BraTS2021 dataset contains 8160 MRI scan data of 2040 patients, of which 1251 cases are used as training sets and publicly annotated, and 219 cases are divided into validation sets, whose annotations are not made public. Each patient has MR images of four modalities: T1, T1ce, T2 and T2-FLAIR. The annotation content of BraTS2021 mainly includes three types: enhanced tumor (ET), peritumoral edema / invasive tissue (ED) and necrotic tumor core (NCR). The BraTS2021 dataset is preprocessed by random flipping, random cropping and denoising, which can increase the diversity of the dataset, improve the robustness and generalization ability of the model, reduce overfitting, and improve image quality and model performance.

[0057] S200. Build a brain tumor segmentation model based on multi-scale feature mining of improved 3D U-Net.

[0058] In this embodiment, the basic 3D U-Net network consists of two parts: an encoder and a decoder. The encoder is used to extract various useful feature information from the input MR images of four modalities. The encoder consists of four downsampling modules (also called convolution blocks), each of which contains two 3D convolution layers, a Batch Normalization layer, and an activation function ReLU layer. The Batch Normalization layer is used to accelerate training and prevent overfitting, and the activation function ReLU layer is used to introduce nonlinearity, so that the network can learn more complex features. Each downsampling module is followed by a 3D maximum pooling layer to reduce the size of the feature map and increase the receptive field, so that the network can capture a wider range of contextual information. Taking the input image resolution of 256x256x128 as an example, the size of the feature map will gradually decrease with the downsampling operation in the feature extraction stage. For example, after the first downsampling module, the size of the feature map becomes 128x128x64 (the pooling kernel size is 2x2x2). The decoder consists of four upsampling modules, each of which is responsible for gradually restoring the size of the feature map to its original size. The upsampling module consists of a 3D transposed convolution (also called deconvolution), two 3D convolution layers, two Batch Normalization layers, and two activation function ReLU layers. In addition, through skip connections, the feature fusion part receives feature maps from the same level of the feature extraction stage and the multi-scale feature mining stage, and fuses them with the upsampling results in a splicing manner to retain more spatial information and high-level semantic information. This fusion method helps to combine low-level detail information with high-level semantic information, thereby improving the accuracy of segmentation. In the final stage of feature fusion, a 1x1x1 convolution kernel is used to reduce the number of channels of the feature map to the same number as the number of categories, and then a sigmoid activation function is applied to generate the final brain tumor segmentation result.

[0059] In order to improve the basic 3D U-Net network model, the present invention introduces a multi-scale feature mining module to improve the 3DU-Net network. The multi-scale feature mining module is composed of the ASPP (Atrous Spatial PyramidPooling) module and the attention mechanism in the DeepLab model. The ASPP module is designed to capture contextual information at different scales in the image, which is crucial for understanding the image content. By using multiple parallel dilated convolution branches, each branch using a different dilated rate, the model can capture multiple scale information from local to global. Dilated convolution is the core operation in the ASPP module, which allows the model to expand the receptive field without increasing the amount of computation. The increase in the receptive field means that the model can "see" a larger range of image content, which is particularly important for identifying large-scale objects in the image. The present invention inputs the 4 layers of features output by the 4 downsampling blocks of the encoder into the ASPP module respectively to obtain contextual information at different scales, which helps to improve the accuracy of brain tumor segmentation.

[0060] S300: training the brain tumor segmentation model using the brain tumor data to be processed, and saving the optimal weight model during the training process.

[0061] In this embodiment, the improved model is trained by first setting hyperparameters, including using the stochastic gradient descent optimization algorithm SGD (Stochastic Gradient Descent), and using the cosine annealing strategy to smoothly adjust the learning rate. The preprocessed brain tumor data is input into the brain tumor segmentation model for training, and the mixed loss function BCEDice Loss is used to calculate the error value between the output prediction value of the brain tumor segmentation model and the true label value, and back propagation is used to update the parameters of the network and save the optimal weight information.

[0062] S400: Use the optimal weight model to perform model prediction and calculate performance indicators.

[0063] In this embodiment, the trained optimal weight model is used to predict the validation set in the multimodal brain tumor data to obtain a predicted image and performance index, wherein the Dice similarity coefficient is used as a performance index for quantitatively evaluating the segmentation effect of the model.

[0064] As an optional implementation scheme of the present application, optionally, in step S100, multimodal brain tumor data is acquired, and the multimodal brain tumor data is preprocessed to obtain the brain tumor data to be processed, including: the multimodal brain tumor data is the public glioma data set BraTS2021 provided by the International Association for Medical Image Computing and Computer-Assisted Intervention; the training set in the multimodal brain tumor data is preprocessed using random flipping, random cropping and denoising to obtain the brain tumor data to be processed.

[0065] In this embodiment, the multimodal brain tumor data is the public glioma dataset BraTS2021 provided by the International Association for Medical Image Computing and Computer-Assisted Intervention. Each image data in the publicly annotated training set in the multimodal brain tumor data contains MR images and label images of four modalities (T1, T1ce, T2 and T2-FLAIR), where the label data can be divided into 4 parts: enhanced tumor region, peritumoral edema region, necrotic tumor core region and background. In order to better evaluate the segmentation, these parts are divided into three segmentation region tasks, including the segmentation of WT (whole tumor region), TC (tumor core region) and ET (enhancing tumor region), where WT includes enhanced tumor region, peritumoral edema region and necrotic tumor core region, TC includes enhanced tumor region and necrotic tumor core region, and ET only includes enhanced tumor region. The publicly annotated training set in the multimodal brain tumor data is subjected to preprocessing operations including random rotation, flipping, cropping and adding Gaussian noise to obtain the brain tumor data to be processed. Data augmentation techniques, including random rotation, flipping, cropping, and adding Gaussian noise to the data, help the model see more data variations during training, thereby improving its generalization ability.

[0066] As an optional implementation scheme of the present application, optionally, in step S200, the constructed improved 3D U-Net multi-scale feature mining brain tumor segmentation model includes three parts: an encoder, a multi-scale feature mining module and a decoder.

[0067] In this implementation scheme, the brain tumor segmentation model framework of the improved 3D U-Net multi-scale feature mining is as follows Figure 3 As shown in the figure, the whole is based on the 3D U-Net architecture, including the encoder part, the multi-scale feature mining part and the decoder part. The encoder part of the brain tumor segmentation model is shown in the figure. Figure 3The left part of the figure contains 5 layers, mainly composed of DoubleConv and MaxPool3d. First, the obtained 4 modal data of brain tumor are input into a layer of DoubleConv structure, and then 4 blocks composed of MaxPool3d and DoubleConv are input in turn for feature extraction and size reduction. The detailed structure of DoubleConv block is shown in Figure 4 As shown in (a), after 1 3×3 convolution layer, 1 BatchNorm3d layer, 1 ReLU layer, 1 3×3 convolution layer, 1 BatchNorm3d layer, and 1 ReLU layer, dual feature extraction is achieved. Figure 3 As shown in the middle part of the figure, it is mainly composed of the ASPP module, the FMM module and the Concat operation, which mines the feature information related to the brain tumor area at multiple levels from the 4-layer feature map extracted from the encoder part. The detailed structure of the ASPP module is shown in Figure 4 As shown in (b), it consists of 5 parallel branches. The top layer uses a 1×1 3D convolution layer to change the number of channels of the input features. The middle three layers use three 3×3 3D convolution layers. The three convolution layers are set with hole rates of 6, 12, and 18 respectively to expand the receptive field, thereby capturing tumors of different sizes. The bottom layer uses a global average pooling layer (AdaptiveAvgPool3d) to convert the input feature map into a single feature vector, which represents the global contextual information of the entire input image. Finally, the output of the 1×1 3D convolution layer, the output of the three hole convolution layers, and the output of the AvgPool3d layer are concatenated in the channel dimension. The concatenated feature map is fused through another 1×1 3D convolution layer to generate the final output feature map. The decoder part is shown in the figure. Figure 3 As shown on the right, it contains a 5-layer structure, mainly composed of DoubleConv and ConvTranspose3d.

[0068] The ConvTranspose3d layer converts a smaller input feature map into a larger output feature map through a transposed convolution operation. First, the fourth and fifth layer features containing multi-scale information extracted by the ASPP module and the FMM module are concat-operated and then input into a decoder block. The first, second, and third layer feature operations are the same as before. After being processed by the 4-layer decoder block, the final feature is input into a 1×1 3d convolution layer to obtain a 4-dimensional feature map. Each dimension corresponds to a tumor segmentation category. Finally, a Sigmoid operation is performed to obtain the final brain tumor segmentation map.

[0069] As an optional implementation scheme of the present application, optionally, in step S300, the brain tumor segmentation model is trained using the brain tumor data to be processed, and the optimal weight model in the training process is saved, including: setting hyperparameters; training the constructed brain tumor segmentation model according to the brain tumor data to be processed; and saving the optimal weight model in the training process.

[0070] In this embodiment, first, hyperparameters are set for the brain tumor segmentation model. For example, the network structure is implemented using the Pytorch framework, the initial learning rate is 0.004, the momentum is set to 0.9, the batchsize is 1, the number of training iterations (eopch) is 200, the optimization method uses SGD, and the learning rate scheduler is cosine annealing. The training set of the multimodal brain tumor data is input into the brain tumor segmentation model for training, and the mixed loss function BCEDice Loss is used to calculate the error between the predicted value output by the brain tumor segmentation model and the true label value, and the various parameters of the brain tumor segmentation model are updated by back propagation and the optimal weight model during the training process is saved.

[0071] As an optional implementation scheme of the present application, optionally, in step S400, the optimal weight model is used to perform model prediction and calculate performance indicators, including: using the optimal weight model to predict the validation set, obtaining and saving the prediction results; and calculating performance indicators based on the prediction results and the true labels.

[0072] In this embodiment, the validation set of the multimodal brain tumor data is input into the optimal weight model for segmentation prediction, the segmentation prediction map of each patient is saved, the segmentation prediction map is compared with the true label, and the performance index is calculated.

[0073] The performance indicator used in this paper is the Dice similarity coefficient, which is used to evaluate the similarity between the segmentation results predicted by the model and the actual segmentation labels. The value range is [0,1] The larger the value, the closer the tumor segmentation result is to the true label result, and the better the segmentation effect. The calculation formula is as follows:

[0074]

[0075] Among them, ∩ is the logical AND operator, P1 is the area of ​​the segmentation result predicted by the model, and T1 represents the actual segmentation area.

[0076] Ablation experiments were conducted on the BraTS2020 dataset. Different modules were added to the 3D U-Net model in turn, and the experiments were performed using the same parameter settings. The specific results are shown in Table 1, where ASPP represents the atrous spatial pyramid pooling module and FMM represents the feature mining module.

[0077] Table 1 Comparison of segmentation results of 3D U-Net with different modules added

[0078]

[0079] It can be seen from Table 1 that after adding the ASPP module or the FMM module on the basis of the 3D U-Net structure, ET, WT and TC are increased by about 1%, 1%, and 0.5%, respectively. After adding the ASPP module and the FMM module at the same time, ET, WT and TC are increased by about 2%, 2%, and 1%, respectively. The experimental results show that the different modules added to the improved network structure based on 3D U-Net in the present invention can improve the detection performance of brain tumor segmentation.

[0080] As an optional implementation scheme of the present application, optionally, the constructed brain tumor segmentation model includes three parts: an encoder, a multi-scale feature mining module and a decoder, including: the multi-scale feature mining module also includes a key feature mining module FMM (Feature mining module).

[0081] In this embodiment, after the ASPP module in the multi-scale feature mining module obtains rich feature information, there is also a problem of complicated feature information. To this end, the present invention designs a key feature mining module FMM, which effectively integrates the spatial attention module and the channel attention module, and can extract more critical and effective information from the input feature map to improve the performance of brain tumor segmentation. The spatial attention module focuses on the importance of each position (pixel point) in the feature map. It does this by generating a spatial attention map of the same size as the input feature map. Each value on this attention map represents the weight of the corresponding position in the feature map. The channel attention module focuses on the importance of each channel (feature map) in the feature map. It generates a channel attention vector with the same number of channels as the input feature map, and each value on this vector represents the weight of the corresponding channel. The spatial attention map and the channel attention vector are fused with the original feature map in an element-by-element multiplication manner. This ensures that important spatial and channel information is emphasized while retaining the original feature information. The detailed structure of the FMM module is as follows Figure 4As shown in (c), a 3×3 convolution layer is first applied to the input features, and then a spatial attention layer is generated through a Sigmoid layer. The spatial attention layer is element-wise multiplied and added with the input features to strengthen the focus on the spatial feature information. After that, the feature is output through an AvgPool3d layer to output a feature map of size 1×1×1×C (C is the number of channel dimensions), and then a 1×1 3d convolution layer, a ReLU layer, a 1×1 3d convolution layer and a Sigmoid layer are used to output a channel attention layer. This layer is then multiplied with the unprocessed feature map to obtain the output feature map, which contains key feature information at both the spatial and channel levels.

[0082] As an optional implementation scheme of the present application, optionally, it is characterized in that, according to the brain tumor data to be processed, the constructed brain tumor segmentation model is trained, including: using a mixed loss function BCEDice Loss to calculate the error value between the predicted value output by the brain tumor segmentation model and the true label value, so as to improve the effect of model training.

[0083] In this embodiment, a hybrid loss function BCEDice Loss is used to calculate the error value between the predicted result of the brain tumor segmentation model and the true label. The hybrid loss function BCEDice Loss combines the binary cross entropy loss function and the Dice coefficient loss function, and is used to deal with unbalanced category problems in tasks such as image segmentation. It can more comprehensively evaluate the difference between the predicted result and the true label and improve the effect of model training.

[0084] The binary cross entropy loss function is calculated as follows:

[0085]

[0086] Where N is the number of batch samples, y i is the true label of the i-th sample, p i is the probability that the model predicts that the i-th sample is a positive sample.

[0087] The Dice coefficient loss function calculation formula is as follows:

[0088]

[0089] Where N is the number of batch samples, y i is the true label of the i-th sample, p i is the probability that the model predicts that the i-th sample is a positive sample.

[0090] Therefore, the calculation formula of the mixed loss function BCEDice Loss is as follows:

[0091] L=αL BCE +(1-α)L Dice

[0092] Among them, α is 0.5

[0093] Example 2

[0094] Based on the same principle as the above method, a brain tumor segmentation device based on improved 3D U-Net is also proposed, see Figure 2 , a brain tumor segmentation device 100 based on improved 3D U-Net according to an embodiment of the present disclosure includes:

[0095] A data acquisition and data preprocessing module 110, for acquiring multimodal brain tumor data and preprocessing the multimodal brain tumor data;

[0096] A model building module 120 is used to build an improved brain tumor segmentation model for multi-scale feature mining;

[0097] A training model module 130 is used to train the constructed brain tumor segmentation model to obtain an optimal weight model;

[0098] The model prediction and calculation module 140 is used to use the optimal weight model to predict the validation set and obtain the prediction results and calculated performance indicators.

[0099] As an optional implementation scheme of the present application, optionally, building a model module includes:

[0100] an encoder module for extracting features from the multimodal brain tumor data;

[0101] A multi-scale mining module is used to obtain contextual information at different scales, which helps to improve the accuracy of brain tumor segmentation;

[0102] The decoder module is used to gradually restore the size of the feature map to its original size to generate the final brain tumor segmentation result.

[0103] Obviously, those skilled in the art should understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned control methods. The modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by computing devices, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules respectively, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0104] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned control methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0105] Example 3

[0106] Furthermore, an electronic device is proposed, comprising:

[0107] processor;

[0108] a memory for storing processor-executable instructions;

[0109] Wherein, the processor is configured to implement the brain tumor segmentation method based on improved 3D U-Net described in Example 1 when executing the executable instructions.

[0110] The electronic device of the embodiment of the present disclosure includes a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement any of the above-mentioned brain tumor segmentation methods based on improved 3D U-Net when executing the executable instructions.

[0111] It should be noted that the number of processors may be one or more. At the same time, the electronic device of the embodiment of the present disclosure may also include an input device and an output device. The processor, memory, input device, and output device may be connected via a bus or in other ways, which are not specifically limited here.

[0112] The memory is a computer-readable storage medium for the brain tumor segmentation method based on the improved 3D U-Net, and can be used to store software programs, computer executable programs, and various modules, such as: the program or module corresponding to the brain tumor segmentation method based on the improved 3D U-Net in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0113] The input device can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output device can include a display device such as a display screen.

[0114] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A brain tumor segmentation method based on improved 3D U-Net, characterized in that By combining deep learning and image processing technology, high-precision and automatic segmentation of brain tumor images is performed, and the method includes the following steps: Acquiring multimodal brain tumor data, and preprocessing the multimodal brain tumor data to obtain brain tumor data to be processed; Construct a brain tumor segmentation model based on multi-scale feature mining of improved 3D U-Net; Using the brain tumor data to be processed to train the brain tumor segmentation model, and saving the optimal weight model during the training process; The optimal weight model is used to perform model prediction and calculate performance indicators.

2. The brain tumor segmentation method based on improved 3D U-Net according to claim 1, characterized in that: Acquiring multimodal brain tumor data and preprocessing the multimodal brain tumor data to obtain brain tumor data to be processed, including: The multimodal brain tumor data is the public glioma dataset BraTS2021 provided by the International Association for Medical Image Computing and Computer-Assisted Intervention; The training set in the multimodal brain tumor data is preprocessed by using random flipping, random cropping, denoising and other methods to obtain the brain tumor data to be processed.

3. The brain tumor segmentation method based on improved 3D U-Net according to claim 1, characterized in that: The brain tumor segmentation model constructed by multi-scale feature mining of improved 3D U-Net includes three parts: encoder, multi-scale feature mining module and decoder.

4. The brain tumor segmentation method based on improved 3D U-Net according to claim 1, characterized in that: The brain tumor segmentation model is trained using the brain tumor data to be processed, and the optimal weight model in the training process is saved, including: Set hyperparameters; Training the constructed brain tumor segmentation model according to the brain tumor data to be processed; Save the optimal weight model during training.

5. The brain tumor segmentation method based on improved 3D U-Net according to claim 1, characterized in that: The optimal weight model is used to perform model prediction and calculate performance indicators, including: Using the optimal weight model to predict the validation set in the multimodal brain tumor data, and obtaining and saving the prediction results; The performance index is calculated based on the predicted results and the true labels.

6. The brain tumor segmentation method based on improved 3D U-Net according to claim 3, characterized in that: The multi-scale feature mining module also includes a key feature mining module FMM.

7. The brain tumor segmentation method based on improved 3D U-Net according to claim 4, characterized in that: The mixed loss function BCEDice Loss is used to calculate the error value between the predicted value output by the brain tumor segmentation model and the true label value, so as to improve the effect of model training.

8. A brain tumor segmentation device based on improved 3D U-Net, characterized in that: include: A data acquisition and data preprocessing module, used for acquiring multimodal brain tumor data and preprocessing the multimodal brain tumor data; Build a model module to build a brain tumor segmentation model based on multi-scale feature mining of improved 3D U-Net; A training model module, used to train the constructed brain tumor segmentation model to obtain an optimal weight model; The model prediction and calculation module is used to use the optimal weight model to predict the multimodal brain tumor data validation set to obtain prediction results and calculated performance indicators.

9. The brain tumor segmentation device based on improved 3D U-Net as claimed in claim 8, characterized in that: Build model modules, including: an encoder module for extracting features from the multimodal brain tumor data; A multi-scale mining module is used to obtain contextual information at different scales, which helps to improve the accuracy of brain tumor segmentation; The decoder module is used to gradually restore the size of the feature map to its original size to generate the final brain tumor segmentation result.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement a brain tumor segmentation method based on an improved 3D U-Net as described in any one of claims 1 to 8 when executing the executable instructions.

Images