Brain MRI tumor segmentation method, device, system, and storage medium

Abstract

The present invention discloses a brain MRI tumor segmentation method, device, system, and storage medium, including: step S1, obtaining a brain MRI image; step S2, preprocessing the brain MRI image; step S3, training a weak edge and context feature fusion model based on the preprocessed brain MRI image; step S4, inputting the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation. The technical solution of the present invention is adopted to achieve accurate and efficient brain MRI tumor segmentation.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a brain MRI tumor segmentation method and device, system, and storage medium. Background Art

[0002] Brain Magnetic Resonance Imaging (MRI) is a safe and non-destructive detection method that can display brain tissue structure and lesion areas, effectively assisting in the diagnosis of clinical medical diseases. In order to improve the accuracy and efficiency of tumor lesion area segmentation in brain MRI, medical image processing methods based on deep learning are gradually being applied to brain MRI tumor segmentation.

[0003] Most of the early brain MRI tumor segmentation methods proposed segmentation algorithms based on region growing and edge features in combination with the characteristics of pixel value transformation. In recent years, with the rapid development of artificial intelligence technology, methods around deep learning and image processing have been widely used in medical diagnosis. However, due to the fuzzy edges and different shapes of tumors in brain MRI, it is difficult for existing edge detection methods to adaptively detect and retain edge detail features. In addition, as the neural network continues to deepen, the segmentation method based on deep learning is prone to lose detailed information such as lesion edges and textures, resulting in poor lesion edge recognition. In particular, image context information is easily obliterated during feature fusion, and the global and local features of the lesion cannot be fully perceived, thus affecting the accuracy of brain tumor segmentation. At the same time, the brain tumor diagnosis process is highly subjective and relies on expert experience. It is difficult to achieve accurate segmentation of brain MRI tumor lesion areas through manual observation alone, and there are disadvantages such as low diagnostic efficiency.

[0004] The patent with publication number CN116523877A discloses a brain MRI image tumor block segmentation method based on convolutional neural network. This method constructs a 3D-U-Net neural network model and optimizes the 3D-U-Net model using a particle swarm optimization algorithm, effectively improving the convergence speed and convergence accuracy of the network. However, the edge information of the lesion area is easily lost, resulting in a slightly insufficient model segmentation accuracy. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a brain MRI tumor segmentation method and device, system, and storage medium, which adaptively detects the lesion edge information of the weak edge of the brain MRI tumor area, constructs an edge branch structure based on a gated loop to transmit the edge context information of the lesion area, and designs a multi-layer feature map maximum index fusion mechanism to highlight the underlying detail features such as the edge, so as to achieve accurate and efficient brain MRI tumor segmentation.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A brain MRI tumor segmentation method, comprising:

[0008] Step S1, obtaining a brain MRI image;

[0009] Step S2, preprocessing the brain MRI image;

[0010] Step S3, training a weak edge and context feature fusion model based on the preprocessed brain MRI image;

[0011] Step S4: input the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation.

[0012] Preferably, step S2 specifically includes: cropping the brain MRI image into images of the same size and performing grayscale processing.

[0013] Preferably, step S3 comprises:

[0014] Step S31, using the Otsu threshold segmentation method to calculate the lesion edge gradient threshold twice on the preprocessed brain MRI image to extract the lesion edge features;

[0015] Step S32, extracting brain MRI context features from the preprocessed brain MRI image through forward propagation based on the edge branch model of GRU;

[0016] Step S33: using the feature edge maximum index fusion mechanism, the brain MRI context features and the lesion edge features are fused according to the index position.

[0017] The present invention also provides a brain MRI tumor segmentation device, comprising:

[0018] An acquisition module, used for acquiring brain MRI images;

[0019] A preprocessing module, used for preprocessing brain MRI images;

[0020] A training module, used for training a weak edge and context feature fusion model based on preprocessed brain MRI images;

[0021] The segmentation module is used to input the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation.

[0022] Preferably, the preprocessing module is used to crop the brain MRI images into images of the same size and perform grayscale processing.

[0023] Preferably, the training module includes:

[0024] The first processing module is used to calculate the lesion edge gradient threshold twice using the Otsu threshold segmentation method on the preprocessed brain MRI image to extract the lesion edge features;

[0025] The second processing module is used for extracting brain MRI context features from the preprocessed brain MRI image through forward propagation based on the GRU edge branch model;

[0026] The fusion module is used to fuse brain MRI context features and lesion edge features according to index positions through a feature edge maximum index fusion mechanism.

[0027] The present invention also provides a brain MRI tumor segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a brain MRI tumor segmentation method when executed by the processor.

[0028] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the brain MRI tumor segmentation method when running.

[0029] The present invention firstly aims at the problem that it is difficult to effectively detect and retain the detailed features of the lesion edge, adopts the Otsu threshold segmentation method to calculate the gradient threshold of the lesion edge, and formulates the weak edge detection rule to realize the adaptive recognition of the detailed features such as the lesion edge; secondly, in view of the problem that the image context information is easily lost when transmitted with the deep neural network, a dual-branch network structure based on the gated recurrent unit (Gated Recurrent Unit, GRU) and the deep convolutional neural network (Convolutional Neural Networks, CNN) is constructed to ensure the effective transmission of the tumor edge context information through forward propagation; finally, in view of the problem that the key information of the lesion is easily obliterated during the feature fusion process, the encoder, decoder and edge feature map are fused according to the index position, and the details and context information are fully balanced to avoid the obliteration of the key features. In order to improve the accuracy and efficiency of brain MRI tumor segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 is a flow chart of the brain MRI tumor segmentation method of the present invention;

[0032] Figure 2 It is a principle diagram of the brain MRI tumor segmentation method of the present invention;

[0033] Figure 3 Schematic diagram of Otsu dual threshold weak edge adaptive detection;

[0034] Figure 4 This is a diagram of the edge context branch architecture based on GRU;

[0035] Figure 5 This is a diagram of the architecture for fusion of the maximum edge index of multi-layer feature maps. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Embodiment 1:

[0039] like Figure 1 and 2 As shown, an embodiment of the present invention provides a brain MRI tumor segmentation method, comprising:

[0040] Step S1, obtaining a brain MRI image, wherein the brain MRI image includes: normal brain medical images and brain tumor-existing brain medical images;

[0041] Step S2, preprocessing the brain MRI image, wherein the preprocessing is to cut the brain MRI image into images of the same size and perform grayscale processing;

[0042] Step S3, training a weak edge and context feature fusion model based on the preprocessed brain MRI image;

[0043] Step S4: input the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation.

[0044] Further, step S3 includes:

[0045] Step S31, using the Otsu threshold segmentation method (Otsu DoublethresholdWeak-edges adaptive detection, Otsu-DW) to calculate the lesion edge gradient threshold twice on the preprocessed brain MRI image to extract the lesion edge features;

[0046] Step S32, constructing a GRU-based Edge Branch model (GRU-EB), extracting brain MRI context features from the preprocessed brain MRI image through forward propagation, and ensuring effective transmission of tumor edge context information;

[0047] Step S33: Through the multi-layer feature channel max-index fusion mechanism (MF-Max), brain MRI context features and lesion edge features are fused according to the index position to fully balance the details and context information to avoid the annihilation of key features.

[0048] As an implementation method of the embodiment of the present invention, Figure 3 As shown, step S31 includes:

[0049] S311: Calculate strong edge gradient value based on Otsu threshold segmentation

[0050] The brain MRI gradient histogram G is solved according to the Canny edge detection method and transformed to the [0,255] interval through linear mapping. Then, the segmentation threshold of the gradient histogram G is calculated using the Otsu threshold segmentation algorithm. Assume that the strong edge gradient threshold is T s , then the gradient histogram can be divided into foreground C f and background C b Two parts. The between-class variance σ of the two categories 2 It can be expressed as shown in formula (1). When it is the largest, the classification effect is the best, that is, T s The value is the best.

[0051]

[0052] S312: Fusion of nonlinear feature mapping and Otsu calculation of weak edge gradient values

[0053] Since the fuzzy boundary of brain MRI tumors can easily lead to confusion with normal brain areas during edge extraction, a S-shaped function is designed to extract the edges of tumors smaller than the strong edge gradient threshold T. s The gradient value of is processed by nonlinear feature mapping, and the calculation method is shown in formula (2).

[0054]

[0055] By transforming the gradient value to the interval [0,255] through nonlinear feature mapping, the weak edge information of the normal area in brain MRI can be suppressed on the one hand, and the gradient value of the tumor edge can be greatly enhanced on the other hand, so as to more effectively extract the features of the lesion area. Similarly, the Otsu threshold segmentation method is used to solve the processed gradient value again, and the result is reversely calculated according to the feature mapping relationship to obtain the weak edge threshold T w .

[0056] S313: Brain MRI tumor weak edge detection

[0057] According to the calculated strong and weak edge gradient threshold T s and T w The non-maximum suppression formula is used to screen the brain MRI gradient values, retaining only strong and weak edges and removing invalid edges. At the same time, the original gradient histogram G is updated to G′ for edge connection according to the strong and weak edge judgment formula (3).

[0058]

[0059] Afterwards, the pixel index positions corresponding to the gradient values ​​that meet the conditions are stored, and a texture map I covering all strong and weak edges is generated according to the size of the input brain MRI image. edges .

[0060] S314: Weak edge connection strategy

[0061] First, the coordinates (x, y) of all weak edge pixels Weak are obtained according to the gradient map G′. Then, all Weaks are traversed in the gradient map G′, and the connectivity analysis is performed in combination with the strong and weak edge pixels, that is, whether the point is retained is determined by judging whether there are Strong and other Weaks in the 16-neighborhood of the Weak point. Finally, after removing the isolated points Weak, only the pixels that meet the rules are retained, and they are connected to form a complete edge image Edges as the output.

[0062] As an implementation method of the embodiment of the present invention, Figure 4 As shown, step S32 includes:

[0063] S321: edge texture image encoding operation by row

[0064] The binary edge image with input value (128, 128) is read by row index. i As the input of the neural network and perform operations, the process of each GRU is shown in equations (4), (5), (6) and (7).

[0065] Ri =σ(p i W xr +H i-1 W hr +b r ) (4)

[0066] Z i =σ(p i W xz +H i-1 W hz +b z ) (5)

[0067]

[0068] Among them, R i Represents the output of the reset gate, which determines whether the network ignores the previous state information, thereby controlling the flow of information. i-1 represents the hidden state of the previous sub-row, W and b represent the learnable weights and biases. σ is the sigmoid activation function. Z i Represents the output of the update gate, which controls whether the model remembers the previous state information and how to combine the new input information with the previous state information. Under the action of the update gate and the reset gate, the candidate hidden state can be calculated Final output H i Indicates the current input line p i The hidden state of .

[0069] S322: Design multi-layer GRU forward propagation calculation

[0070] The output of the corresponding encoder defines a 4-layer GRU neural network structure, which extracts and retains edge context information features in depth, effectively avoiding the limitations of CNN when processing context information. The propagation calculation method of each layer of GRU is shown in formula (8).

[0071] H i =L GRU (H i-1 ,p i ) (8)

[0072] Among them, H i is the output result of the pixel value of the current sub-row, H i-1 is the previous output state, L GRU The representative input will be calculated by each layer of GRU neurons. In this model, the input sub-row size of each layer of GRU is 128, the number of hidden layer neurons is 64, the output size is 128, and each layer of GRU contains 1 hidden layer calculation.

[0073] S323: Reverse update based on loss function

[0074] According to the output of the multi-layer GRU, the size is re-transformed and converted to the input size (128, 128). Since the structure is to retain edge information as much as possible, the output result corresponds to the edge pixels of the brain tumor segmentation result, and the Binary Cross Entropy Loss is used to update the network parameters of the module. The loss function is defined as shown in formula (9).

[0075]

[0076] Where N is the number of samples, y i is the true label (0 or 1) of the i-th sample, is the predicted probability of the i-th sample. Loss1 is only used for the recurrent neural network branch to learn edge context information to avoid the obliteration of detailed features during deep convolution.

[0077] As an implementation method of the embodiment of the present invention, Figure 5 As shown, step S33 is:

[0078] S331: Integrating the dimensions of feature maps from different network layers

[0079] The 4-layer feature map output by GRU is expanded and max-pooled to keep it consistent with the dimensions of the feature map output by the first 4 layers of the encoder. The unified feature map is easier to integrate and compare features.

[0080] The processing process is shown in formula (10).

[0081]

[0082] S332: Fusion of the corresponding output features of GRU and encoder according to the maximum index value

[0083] For the two sets of feature maps of the i-th layer with the same dimension size after expansion and Compare each layer separately and save only the features with the maximum value according to the index order Finally, it is spliced ​​with the original encoder features to obtain a preliminary fused feature map The processing process is shown in equations (11) and (12).

[0084]

[0085] S333: Deeply fuse edge features with decoder feature maps

[0086] The edge feature map containing edge information is concatenated and fused with the decoder output feature map layer by layer. The fusion process is shown in formula (13).

[0087]

[0088] The embodiment of the present invention adopts Otsu Double threshold Weak-edges adaptive detection (Otsu-DW), based on the GRU-based Edge Branch model (GRU-EB) and the Multi-layer Feature Channel Max-index fusion mechanism (MF-Max) to achieve the segmentation of brain MRI tumors; wherein, the Otsu-DW automatically selects the strong and weak edge thresholds in the brain MRI image through two Otsu threshold segmentations, corrects the error existing in the Canny manual threshold setting, especially in the second threshold segmentation, designs an S-type function to enhance the weak edge gradient value to facilitate the distinction between lesions and normal brain sulci; compared with Canny, the weak edge connection rules formulated by Otsu-DW can effectively retain the weak edge pixels of brain tumors. The GRU-EB combines the traditional threshold segmentation method with deep learning model training, and introduces a multi-layer GRU neural network in the branch network to forward propagate the edge image of brain MRI; compared with local feature extraction methods such as CNN and attention mechanism, GRU-EB focuses more on the long-sequence semantic feature association of the image as a whole while capturing spatial correlation, thereby better transmitting the contextual information of the lesion area features. The MF-Max relies on the encoder-decoder structure to design jump connections to enhance the underlying detail information of brain MRI lesions, and combines the GRU-EB branch feature map with the deep semantic feature comparison fusion to retain the key edge context features of the tumor; compared with feature fusion methods such as direct splicing or convolution of feature maps, MF-Max can better balance the underlying details and deep semantic information, thereby improving the segmentation accuracy of brain MRI tumors.

[0089] Embodiment 2:

[0090] The embodiment of the present invention further provides a brain MRI tumor segmentation device, comprising:

[0091] An acquisition module, used for acquiring brain MRI images;

[0092] A preprocessing module, used for preprocessing brain MRI images;

[0093] A training module, used for training a weak edge and context feature fusion model based on preprocessed brain MRI images;

[0094] The segmentation module is used to input the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation.

[0095] As an implementation of the embodiment of the present invention, the preprocessing module is used to crop the brain MRI image into images of the same size and perform grayscale processing.

[0096] As an implementation of an embodiment of the present invention, the training module includes:

[0097] The first processing module is used to calculate the lesion edge gradient threshold twice using the Otsu threshold segmentation method on the preprocessed brain MRI image to extract the lesion edge features;

[0098] The second processing module is used for extracting brain MRI context features from the preprocessed brain MRI image through forward propagation based on the GRU edge branch model;

[0099] The fusion module is used to fuse brain MRI context features and lesion edge features according to index positions through a feature edge maximum index fusion mechanism.

[0100] Embodiment 2:

[0101] An embodiment of the present invention further provides a brain MRI tumor segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a brain MRI tumor segmentation method when executed by the processor.

[0102] Embodiment 3:

[0103] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is run, the brain MRI tumor segmentation method is executed.

[0104] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A brain MRI tumor segmentation method, characterized in that: include: Step S1, obtaining a brain MRI image; Step S2, preprocessing the brain MRI image; Step S3, training a weak edge and context feature fusion model based on the preprocessed brain MRI image; Step S4, inputting the brain MRI image to be processed into the trained weak edge and context feature fusion model to perform brain MRI tumor segmentation; Step S2 specifically includes: cropping the brain MRI image into images of the same size and performing grayscale processing; Step S3 includes: Step S31, using the Otsu threshold segmentation method to calculate the lesion edge gradient threshold twice on the preprocessed brain MRI image to extract the lesion edge features; Step S32, extracting brain MRI context features from the preprocessed brain MRI image through forward propagation based on the edge branch model of GRU; Step S33: using the feature edge maximum index fusion mechanism, the brain MRI context features and the lesion edge features are fused according to the index position.

2. A brain MRI tumor segmentation device, characterized in that: include: An acquisition module, used for acquiring brain MRI images; A preprocessing module, used for preprocessing brain MRI images; A training module, used for training a weak edge and context feature fusion model based on preprocessed brain MRI images; A segmentation module is used to input the brain MRI image to be processed into a trained weak edge and context feature fusion model to perform brain MRI tumor segmentation; The preprocessing module is used to crop the brain MRI images into images of the same size and perform grayscale processing; The training modules include: The first processing module is used to calculate the lesion edge gradient threshold twice using the Otsu threshold segmentation method on the preprocessed brain MRI image to extract the lesion edge features; The second processing module is used for extracting brain MRI context features from the preprocessed brain MRI image through forward propagation based on the GRU edge branch model; The fusion module is used to fuse brain MRI context features and lesion edge features according to index positions through a feature edge maximum index fusion mechanism.

3. A brain MRI tumor segmentation system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the brain MRI tumor segmentation method according to claim 1 is executed.

4. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program executes the brain MRI tumor segmentation method according to claim 1 when running.

Images