A fetal MRI brain tissue segmentation method and device based on deep learning

Abstract

The present invention relates to the field of medical MRI imaging, and more specifically to a deep learning-based fetal MRI brain tissue segmentation method and device. The method and device first perform data enhancement operations on fetal brain MRI, and then construct a feature pyramid model based on the Contextual Transformer block: the feature pyramid model introduces an attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix and enhance the features extracted from the fetal brain MRI image after data enhancement; the feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field and retains detailed spatial information, and effectively extracts global context information in the medical image, thereby effectively improving the accuracy of segmentation and helping doctors to make clinical diagnoses to the greatest extent.

Description

Technical Field

[0001] The present invention relates to the field of medical MRI imaging, and in particular to a fetal MRI brain tissue segmentation method and device based on deep learning. Background Art

[0002] Congenital diseases are one of the leading causes of neonatal mortality worldwide. Prenatal maternal and fetal health examinations and quantitative analysis of the developing human fetal brain are crucial for a comprehensive understanding of neurodevelopment in both normal and congenitally ill fetuses, as well as for early detection and treatment of congenital diseases. Ultrasound examination is an important tool for prenatal identification and screening of fetal malformations. However, due to the effects of fetal position and cranial acoustic attenuation on ultrasound, ultrasound often has limitations in detecting developmental abnormalities in certain tissues, such as the cerebellar vermis. In recent years, fetal magnetic resonance imaging (MRI) has become an important adjunct tool, providing additional information about fetal development when ultrasound images are unclear, and enabling more accurate diagnosis of congenital diseases. Fetal MRI volumetric and morphological analysis relies on segmentation of the fetal brain into distinct tissue classes. Many congenital diseases cause subtle changes in these tissues, such as Dandy-Walker malformation, which manifests as absent cerebellar vermis and enlarged fourth ventricle. However, manual segmentation is tedious, time-consuming, and error-prone, requiring highly specialized clinicians. Due to the difficulty of segmentation, numerous automated and semi-automated methods have been explored to detect and segment brain tissue to aid diagnosis.

[0003] Atlas-based segmentation techniques rely on an image correction process, which involves finding voxel-to-voxel alignment between two images. For automated segmentation of fetal brain tissue within reconstructed MR volumes, Habas et al. published an article in the journal Human Brain Mapping titled "Atlas-based segmentation of developing tissues in the human brain with quantitative validation in young fetuses." They proposed using an atlas-based expectation maximization (EM) model to segment white matter (WM), gray matter (GM), germinal matrix, and right cerebral spinal fluid (eCSF). Prior to segmentation, another EM model was used for bias field correction.

[0004] At the same time, in recent years, deep learning methods, especially convolutional neural networks (CNNs), have become popular in some target recognition and biological image segmentation challenges. Unlike traditional classification methods, CNNs automatically learn complex and representative features directly from the data itself, eliminating the need to first extract a set of handcrafted features from the image as input to the classifier or model. Due to this characteristic, research on fetal brain tissue segmentation based on CNNs has mainly focused on the design of network architecture rather than feature extraction in image processing. Therefore, deep learning methods often achieve better performance than traditional machine learning methods. Khalili et al. published an article in the journal Magnetic Resonance Imaging titled "Automatic brain tissue segmentation in fetal MRI using convolutional neural networks." They first used a CNN to extract the intracranial volume, automatically cropped the image to the region of interest, and then used a CNN with the same structure to segment the extracted volume into seven brain tissue categories.

[0005] In summary, the above-mentioned prior art has the following technical defects:

[0006] 1. The fetal brain is small, with irregular shapes and complex interconnected tissues;

[0007] 2. Spontaneous movement of the fetus and mother can cause some artifacts, making it difficult to distinguish tissue types and resulting in poor segmentation accuracy in medical MRI images. Summary of the Invention

[0008] The embodiments of the present invention provide a fetal MRI brain tissue segmentation method and device based on deep learning, so as to at least solve the technical problem of poor accuracy of existing medical MRI image segmentation.

[0009] According to one embodiment of the present invention, a deep learning-based fetal MRI brain tissue segmentation method is provided, comprising the following steps:

[0010] Perform data augmentation on fetal brain MRI;

[0011] Constructing a feature pyramid model based on the Contextual Transformer block: The feature pyramid model introduces an attention structure, CoT-Block, in both the encoder and decoder. This CoT-Block uses key context information to guide the learning of a dynamic attention matrix, enhancing the extraction of features from data-augmented fetal brain MRI images. The feature pyramid model also introduces a hybrid dilated convolution module in the decoder. This module expands the receptive field while preserving detailed spatial information, effectively extracting global contextual information from medical images.

[0012] Tissue segmentation of data-augmented fetal brain MRI images was performed using a feature pyramid model.

[0013] Furthermore, data augmentation operations on fetal brain MRI include:

[0014] Adversarial data augmentation technology is used to train a neural network by simulating the intensity inhomogeneity caused by common artifacts in MRI, and the trained neural network is used to perform data augmentation operations on fetal brain MRI.

[0015] Furthermore, adversarial data augmentation techniques are used to train the neural network by simulating the intensity inhomogeneity caused by common artifacts in MRI, including:

[0016] Given a segmentation network f seg (.; θ) and the input image I, first find the optimal value of the control point c in the search space, and construct an adversarial bias field I for the input image bias =G bias (I; c) The neural network then outputs the original prediction p = f seg (I;θ) and the predicted value after perturbation p*=f seg (G bias (I; c); θ), calculate D by the following formula (2) comp ; where D comp represents a composite distance function used to enhance the discrimination between two predictions in segmentation;

[0017] First find the optimal value of the control point c in the constructed bias field, and then optimize the parameter θ in the neural network so that the original prediction and the prediction f after generating the adversarial bias are the same. seg (G bias (I;c adv ); the distance between θ is minimized;

[0018]

[0019] D comp (p,p * )=DKL [p||p * ]+wD contour (p,p * ) (2)

[0020]

[0021] Among them, I represents the input image, θ represents the network parameters used for training, c is the control point used to construct an optimal bias field, fseg(.;θ) represents the segmentation network, Gbias(I;c) is an adversarial bias field, Dcomp represents the composite distance function, p represents the original prediction output by the neural network, p* represents the perturbed prediction output after adding the bias field, DKL represents the KL divergence, Dcontour represents a contour-based loss function, w controls the relative importance between the two equations, M represents the foreground channel, and Sx,y represents the edge extraction of two sobel filters in the x and y directions.

[0022] Furthermore, the feature pyramid model consists of a backbone network, which consists of four parts: encoder, decoder, skip connection and feature combination; the structure formed by connecting each feature map in the decoder with the corresponding feature map from the encoder is a skip connection; the skip connection connects the low-level features of the encoder with the high-level features.

[0023] Furthermore, the encoder and decoder are configured to: introduce an attention structure CoT-Block for feature extraction and utilization; the attention structure CoT-Block is used to utilize the verification code key context information to guide the learning of the dynamic attention matrix.

[0024] Furthermore, the attention structure CoT-Block is configured as follows: first, a kxkxk convolution operation is used on the verification code key to encode the context information between neighbors, then the query and the key after the context information modeling are concat, and then two consecutive 1x1x1 convolutions are used to obtain the global context information. Finally, the output result is obtained by fusing the context information between neighbors with the global context information.

[0025] According to another embodiment of the present invention, a deep learning-based fetal MRI brain tissue segmentation device is provided, comprising:

[0026] A data enhancement unit, used to perform data enhancement operations on fetal brain MRI;

[0027] The feature pyramid model construction unit is used to build a feature pyramid model based on the Contextual Transformer block. The feature pyramid model introduces the attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix and enhance the feature extraction of fetal brain MRI images after data enhancement. The feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field while retaining detailed spatial information, and effectively extracts global context information from medical images.

[0028] The tissue segmentation unit is used to perform tissue segmentation on the data-enhanced fetal brain MRI image using a feature pyramid model.

[0029] A storage medium storing a program file capable of implementing any of the above-mentioned deep learning-based fetal MRI brain tissue segmentation methods.

[0030] A processor is used to run a program, wherein when the program is run, any one of the above-mentioned deep learning-based fetal MRI brain tissue segmentation methods is executed.

[0031] The deep learning-based fetal MRI brain tissue segmentation method and device in the embodiment of the present invention constructs a feature pyramid model based on the Contextual Transformer block: the feature pyramid model introduces the attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix, and enhances the extraction of features in the fetal brain MRI image after data enhancement; the feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field and retains detailed spatial information, and effectively extracts global context information in the medical image, thereby effectively improving the accuracy of the segmentation of each tissue in the fetal brain MRI image, and helping doctors to make clinical diagnoses to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0033] Figure 1 This is the overall block diagram of fetal MRI brain tissue segmentation using deep learning methods in the present invention;

[0034] Figure 2 This is a data enhancement flow chart in the present invention;

[0035] Figure 3 This is a feature pyramid model diagram based on the Contextual Transformer block in the present invention;

[0036] Figure 4 This is the structural diagram of the Contextual Transformer block in the present invention;

[0037] Figure 5 This is the coronal segmentation result diagram of the experimental results of the technical solution of the present invention;

[0038] Figure 6 The following is a comparison chart of indicators of different implementation technologies. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Example 1

[0042] According to one embodiment of the present invention, a deep learning-based fetal MRI brain tissue segmentation method is provided, comprising the following steps:

[0043] Perform data augmentation on fetal brain MRI;

[0044] Constructing a feature pyramid model based on the Contextual Transformer block: The feature pyramid model introduces an attention structure, CoT-Block, in both the encoder and decoder. This CoT-Block uses key context information to guide the learning of a dynamic attention matrix, enhancing the extraction of features from data-augmented fetal brain MRI images. The feature pyramid model also introduces a hybrid dilated convolution module in the decoder. This module expands the receptive field while preserving detailed spatial information, effectively extracting global contextual information from medical images.

[0045] Tissue segmentation of data-augmented fetal brain MRI images was performed using a feature pyramid model.

[0046] The deep learning-based fetal MRI brain tissue segmentation method in an embodiment of the present invention constructs a feature pyramid model based on the Contextual Transformer block: the feature pyramid model introduces the attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix, and enhances the extraction of features in the fetal brain MRI image after data enhancement; the feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field and retains detailed spatial information, and effectively extracts global context information in medical images, thereby effectively improving the accuracy of segmentation and helping doctors to make clinical diagnoses to the greatest extent.

[0047] Among them, the data enhancement operations for fetal brain MRI include:

[0048] Adversarial data augmentation technology is used to train a neural network by simulating the intensity inhomogeneity caused by common artifacts in MRI, and the trained neural network is used to perform data augmentation operations on fetal brain MRI.

[0049] Among them, the use of adversarial data enhancement technology to train the neural network by simulating the intensity inhomogeneity caused by common artifacts in MRI includes:

[0050] Given a segmentation network f seg (.; θ) and the input image I, first find the optimal value of the control point c in the search space, and construct an adversarial bias field I for the input image bias =G bias (I; c) The neural network then outputs the original prediction p = f seg (I;θ) and the predicted value after perturbation p*=f seg (G bias (I; c); θ), Dcomp is calculated by the following formula (2); where Dcomp represents a composite distance function used to enhance the discrimination between two predictions in segmentation;

[0051] First find the optimal value of the control point c in the constructed bias field, and then optimize the parameter θ in the neural network so that the original prediction and the prediction f after generating the adversarial bias are the same. seg (G bias (I;c adv ); the distance between θ is minimized;

[0052]

[0053] D comp (p,p * )=D KL [p||p * ]+wD contour (p,p * ) (2)

[0054]

[0055] Among them, I represents the input image, θ represents the network parameters used for training, c is the control point used to construct an optimal bias field, fseg(.;θ) represents the segmentation network, Gbias(I;c) is an adversarial bias field, Dcomp represents the composite distance function, p represents the original prediction output by the neural network, p* represents the perturbed prediction output after adding the bias field, DKL represents the KL divergence, Dcontour represents a contour-based loss function, w controls the relative importance between the two equations, M represents the foreground channel, and Sx,y represents the edge extraction of two sobel filters in the x and y directions.

[0056] Among them, the feature pyramid model consists of a backbone network, which consists of four parts: encoder, decoder, skip connection and feature combination; the structure formed by connecting each feature map in the decoder with the corresponding feature map from the encoder is a skip connection; the skip connection connects low-level features with high-level features to obtain more accurate spatial information.

[0057] Among them, the attention structure CoT-Block is introduced in both the encoder and decoder parts to realize feature extraction and utilization; the attention structure CoT-Block makes full use of key context information to guide the learning of dynamic attention matrix, thereby enhancing the ability of visual representation.

[0058] Among them, the attention structure CoT-Block is configured as follows: first, a kxkxk convolution operation is used on the key to encode the context information between neighbors, then the query and the key after the context information is modeled are concat, and then two consecutive 1x1x1 convolutions are used to obtain the global context information. Finally, the output result is obtained by fusing the context information between neighbors with the global context information.

[0059] Among them, the feature pyramid model introduces a hybrid dilated convolution module in the last layer of the decoder; the hybrid dilated convolution module increases the receptive field of the convolution kernel while keeping the number of parameters unchanged, so that each convolution output contains a wide range of information, while ensuring that the size of the output feature map remains unchanged.

[0060] The following is a detailed description of the fetal MRI brain tissue segmentation method based on deep learning of the present invention using a specific embodiment:

[0061] In view of the shortcomings of the prior art, in order to solve the problem that manual segmentation of fetal brain tissue is tedious, time-consuming and error-prone, the present invention designs a fetal MRI brain tissue segmentation method based on deep learning, and proposes a feature pyramid model based on ContextualTransformer block. The model introduces a new attention structure CoT-Block in the encoder and decoder parts. The attention structure makes full use of key context information to guide the learning of the dynamic attention matrix, thereby enhancing the ability of feature extraction. At the same time, a hybrid dilated convolution (HDC) module is introduced in the decoder part. The hybrid dilated convolution module can expand the receptive field and retain detailed spatial information, effectively extracting global context information in medical images, thereby effectively improving the accuracy of segmentation and helping doctors to make clinical diagnoses to the greatest extent. The purpose of the present invention is to use an automatic segmentation method to train a deep learning model with fewer model parameters but high segmentation accuracy, and successfully segment and distinguish seven types of fetal brain tissue.

[0062] The technical solution of the present invention is described in detail as follows:

[0063] Using deep learning methods to perform fetal MRI brain tissue segmentation, a feature pyramid model based on Contextual Transformerblock is proposed. The overall block diagram is as follows Figure 1 The specific steps are as follows:

[0064] Step 1: First, perform data augmentation on the fetal brain MRI to improve the generalization ability and robustness of the feature pyramid model training and alleviate the problem of data scarcity.

[0065] To address the problem of artifacts such as intensity unevenness generated during the scanning process, an adversarial data enhancement technology is used to train the neural network by simulating the intensity unevenness (bias field) caused by common artifacts in MRI. This can improve the generalization ability and robustness of the feature pyramid model and alleviate the problem of data scarcity. The flowchart is as follows Figure 2 shown.

[0066] Given a segmentation network f seg (.; θ) and the input image I, first find the optimal value of the control point c in the search space of the neural network (apply the projected gradient descent algorithm for iterative update), and construct an adversarial bias field I for the input image bias =G bias (I; c) The neural network then outputs the original prediction p = f seg (I;θ) and the predicted value after perturbation p*=f seg (G bias (I; c); θ), through D comp Measure (expressed by formula (2)), D comp Represents a composite distance function, which is used to enhance the ability to distinguish between two predictions in segmentation. First, find the optimal value of the control point c in the deviation field. At this time, the parameters in the neural network are fixed, so that D comp Reach the maximum, and then optimize the parameters θ in the neural network so that the original prediction and the prediction after generating the adversarial deviation f seg (G bias (I;c adv ); the distance between θ) is minimized (ultimately making the loss function reach the minimum). L seg represents the cross entropy loss function.

[0067]

[0068] D comp (p,p * )=D KL [p||p * ]+wD contour (p,p * ) (2)

[0069]

[0070] Step 2: Propose a feature pyramid model based on the Contextual Transformer block.

[0071] The overall structure of the feature pyramid model is as follows Figure 3As shown in Figure 2, the feature pyramid model consists of a backbone network consisting of four parts: an encoder, a decoder, skip connections, and feature combination. Each feature map in the decoder is connected to the corresponding feature map from the encoder. This structure is called a skip connection. It connects low-level features with high-level features, thereby obtaining more accurate spatial information. Both the encoder and decoder incorporate the attention structure CoT-Block to extract and utilize features.

[0072] Contextual Transformer block (CoT-Block) is a novel attention structure that makes full use of key context information to guide the learning of dynamic attention matrix, thereby enhancing the ability of visual representation. Its structure is shown in the figure Figure 4 As shown in the figure, a kxkxk convolution operation is first performed on the key to encode the neighboring context information. Then, the query and the key after the context information modeling are concatinated. Two consecutive 1x1x1 convolutions are then used to obtain the global context information. Finally, the output result is obtained by fusing the neighboring context information with the global context information.

[0073] In the last layer of the decoder, a hybrid dilated convolution module is also introduced. This hybrid dilated convolution module increases the receptive field of the convolution kernel while keeping the number of parameters unchanged, so that each convolution output contains a wider range of information; at the same time, it can ensure that the size of the output feature map remains unchanged.

[0074] The key points and intended protection points of the present invention are at least:

[0075] 1. Using an adversarial data augmentation technique to train the neural network by simulating the intensity inhomogeneity (bias field) caused by common artifacts in MRI, the generalization ability and robustness of the feature pyramid model can be improved, and the problem of data scarcity can be alleviated.

[0076] 2. A feature pyramid model based on the Contextual Transformer block is proposed to enhance the network feature extraction capability and effectively extract the global context information of fetal brain images.

[0077] The beneficial effects of the present invention are at least:

[0078] Compared with the existing technology, the specific gains of the present invention are reflected in: the introduction of CoT-Block and hybrid dilated convolution modules to optimize the basic model. Existing research mostly uses the U-Net model, but the traditional U-Net model has several obvious disadvantages, such as insufficient feature extraction capabilities and the failure to use global context information.

[0079] The experimental results of the technical solution of the present invention are as follows:

[0080] Figure 5 This is the coronal segmentation result diagram of the experimental results of the technical solution of the present invention. Figure 5 (a) is the original image, Figure 5 (b) is the label map, Figure 5 (c) is the predicted graph. Figure 6 This is a comparison chart of indicators of different implementation technologies, where Dice is a measure of the similarity between two sets; Precision is the precision rate, which indicates how many examples the model correctly judges as positive examples; Recall is the recall rate, which indicates the proportion of positive examples correctly judged by the model to all positive examples in the dataset.

[0081] From the above two Figure 5-6 It can be seen that the technical solution of the present invention can effectively make up for the defects of the traditional U-Net network, effectively enhance the effect of image segmentation, and help doctors make better clinical diagnoses.

[0082] The modified designs or alternatives of the present invention are as follows:

[0083] 1. In addition to simulating the intensity inhomogeneity (bias field) caused by common MRI artifacts for data augmentation, other data augmentation methods can also be used to remove artifacts and optimize data;

[0084] 2. In addition to using the feature pyramid model based on CoT-Block and HDC, other algorithms can also be used to optimize the model and improve the accuracy of segmentation.

[0085] Example 2

[0086] According to another embodiment of the present invention, a deep learning-based fetal MRI brain tissue segmentation device is provided, comprising:

[0087] A data enhancement unit, used to perform data enhancement operations on fetal brain MRI;

[0088] The feature pyramid model construction unit is used to build a feature pyramid model based on the Contextual Transformer block. The feature pyramid model introduces the attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix and enhance the feature extraction of fetal brain MRI images after data enhancement. The feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field while retaining detailed spatial information, and effectively extracts global context information from medical images.

[0089] The tissue segmentation unit is used to perform tissue segmentation on the data-enhanced fetal brain MRI image using a feature pyramid model.

[0090] The deep learning-based fetal MRI brain tissue segmentation device in the embodiment of the present invention constructs a feature pyramid model based on the Contextual Transformer block: the feature pyramid model introduces the attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix, and enhances the extraction of features in the fetal brain MRI image after data enhancement; the feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field and retains detailed spatial information, and effectively extracts global context information in medical images, thereby effectively improving the accuracy of segmentation and helping doctors to make clinical diagnoses to the greatest extent.

[0091] The following is a detailed description of the fetal MRI brain tissue segmentation device based on deep learning of the present invention using a specific embodiment:

[0092] In view of the shortcomings of the prior art and to solve the problem that manual segmentation of fetal brain tissue is tedious, time-consuming and error-prone, the present invention designs a fetal MRI brain tissue segmentation device based on deep learning, and proposes a feature pyramid model based on ContextualTransformer block. The model introduces a new attention structure CoT-Block in the encoder and decoder parts. The attention structure makes full use of key context information to guide the learning of the dynamic attention matrix, thereby enhancing the ability of feature extraction. At the same time, a hybrid dilated convolution (HDC) module is introduced in the decoder part. The hybrid dilated convolution module can expand the receptive field and retain detailed spatial information, effectively extracting global context information in medical images, thereby effectively improving the accuracy of segmentation and helping doctors to make clinical diagnoses to the greatest extent. The purpose of the present invention is to use an automatic segmentation method to train a deep learning model with fewer model parameters but high segmentation accuracy, and successfully segment and distinguish seven types of fetal brain tissue.

[0093] The technical solution of the present invention is described in detail as follows:

[0094] Using deep learning methods to perform fetal MRI brain tissue segmentation, a feature pyramid model based on Contextual Transformerblock is proposed. The overall block diagram is as follows Figure 1 The specific steps are as follows:

[0095] Step 1: The data augmentation unit first performs data augmentation on the fetal brain MRI to improve the generalization ability and robustness of the feature pyramid model training and alleviate the problem of data scarcity.

[0096] To address the problem of artifacts such as intensity unevenness generated during the scanning process, an adversarial data enhancement technology is used to train the neural network by simulating the intensity unevenness (bias field) caused by common artifacts in MRI. This can improve the generalization ability and robustness of the feature pyramid model and alleviate the problem of data scarcity. The flowchart is as follows Figure 2 shown.

[0097] Given a segmentation network f seg (.; θ) and the input image I, first find the optimal value of the control point c in the search space (apply the projected gradient descent algorithm for iterative update), and construct an adversarial bias field I for the input image bias =G bias (I; c) The neural network then outputs the original prediction p = f seg (I;θ) and the predicted value after perturbation p*=f seg (G bias (I; c); θ), through D compMeasure (expressed by formula (2)), D comp Represents a composite distance function, which is used to enhance the ability to distinguish between two predictions in segmentation. First, find the optimal value of the control point c in the deviation field. At this time, the parameters in the neural network are fixed, so that D comp Reach the maximum, and then optimize the parameters θ in the neural network so that the original prediction and the prediction after generating the adversarial deviation f seg (G bias (I;c adv ); the distance between θ) is minimized (ultimately making the loss function reach the minimum). L seg represents the cross entropy loss function.

[0098]

[0099] D comp (p,p * )=D KL [p||p * ]+wD contour (p,p * ) (2)

[0100]

[0101] Step 2: Feature pyramid model construction unit, a feature pyramid model based on Contextual Transformer block is proposed.

[0102] The overall structure of the feature pyramid model is as follows Figure 3 As shown in Figure 2, the feature pyramid model consists of a backbone network consisting of four parts: an encoder, a decoder, skip connections, and feature combination. Each feature map in the decoder is connected to the corresponding feature map from the encoder. This structure is called a skip connection. It connects low-level features with high-level features, thereby obtaining more accurate spatial information. Both the encoder and decoder incorporate the attention structure CoT-Block to extract and utilize features.

[0103] Contextual Transformer block (CoT-Block) is a novel attention structure that makes full use of key context information to guide the learning of dynamic attention matrix, thereby enhancing the ability of visual representation. Its structure is shown in the figure Figure 4As shown in the figure, a kxkxk convolution operation is first performed on the key to encode the neighboring context information. Then, the query and the key after the context information modeling are concatinated. Two consecutive 1x1x1 convolutions are then used to obtain the global context information. Finally, the output result is obtained by fusing the neighboring context information with the global context information.

[0104] In the last layer of the decoder, a hybrid dilated convolution module is also introduced. This hybrid dilated convolution module increases the receptive field of the convolution kernel while keeping the number of parameters unchanged, so that each convolution output contains a wider range of information; at the same time, it can ensure that the size of the output feature map remains unchanged.

[0105] The key points and intended protection points of the present invention are at least:

[0106] 1. Using an adversarial data augmentation technique to train the neural network by simulating the intensity inhomogeneity (bias field) caused by common artifacts in MRI, the generalization ability and robustness of the feature pyramid model can be improved, and the problem of data scarcity can be alleviated.

[0107] 2. A feature pyramid model based on the Contextual Transformer block is proposed to enhance the network feature extraction capability and effectively extract the global context information of fetal brain images.

[0108] The beneficial effects of the present invention are at least:

[0109] Compared with the existing technology, the specific gains of the present invention are reflected in: the introduction of CoT-Block and hybrid dilated convolution modules to optimize the basic model. Existing research mostly uses the U-Net model, but the traditional U-Net model has several obvious disadvantages, such as insufficient feature extraction capabilities and the failure to use global context information.

[0110] The experimental results of the technical solution of the present invention are as follows:

[0111] Figure 5 This is the coronal segmentation result diagram of the experimental results of the technical solution of the present invention. Figure 5 (a) is the original image, Figure 5 (b) is the label map, Figure 5 (c) is the predicted graph. Figure 6This is a comparison chart of indicators of different implementation technologies, where Dice is a measure of the similarity between two sets; Precision is the precision rate, which indicates how many examples the model correctly judges as positive examples; Recall is the recall rate, which indicates the proportion of positive examples correctly judged by the model to all positive examples in the dataset.

[0112] From the above two Figure 5-6 It can be seen that the technical solution of the present invention can effectively make up for the defects of the traditional U-Net network, effectively enhance the effect of image segmentation, and help doctors make better clinical diagnoses.

[0113] The modified designs or alternatives of the present invention are as follows:

[0114] 1. In addition to simulating the intensity inhomogeneity (bias field) caused by common MRI artifacts for data augmentation, other data augmentation methods can also be used to remove artifacts and optimize data;

[0115] 2. In addition to using the feature pyramid model based on CoT-Block and HDC, other algorithms can also be used to optimize the model and improve the accuracy of segmentation.

[0116] Example 3

[0117] A storage medium storing a program file capable of implementing any of the above-mentioned deep learning-based fetal MRI brain tissue segmentation methods.

[0118] Example 4

[0119] A processor is used to run a program, wherein when the program is run, any one of the above-mentioned deep learning-based fetal MRI brain tissue segmentation methods is executed.

[0120] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0121] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0123] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0124] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A fetal MRI brain tissue segmentation method based on deep learning, characterized in that: The following steps are involved: Perform data augmentation on fetal brain MRI; Constructing a feature pyramid model based on the Contextual Transformer block: This feature pyramid model introduces an attention structure CoT-Block in the encoder and decoder parts. The CoT-Block uses key context information to guide the learning of a dynamic attention matrix and enhance the features extracted from the data-augmented fetal brain MRI images. The feature pyramid model introduces a hybrid dilated convolution module in the decoder part, which expands the receptive field and retains detailed spatial information, and effectively extracts global context information in medical images; Using the feature pyramid model to perform tissue segmentation on the data-enhanced fetal brain MRI image; The data enhancement operation for the fetal brain MRI includes: Using adversarial data augmentation technology, a neural network is trained by simulating the intensity inhomogeneity caused by common artifacts in MRI. The trained neural network is then used to perform data augmentation on fetal brain MRI. The method of using adversarial data enhancement technology to train a neural network by simulating intensity inhomogeneity caused by common artifacts in MRI includes: Given a segmentation network f seg (.; θ) and the input image I, first find the optimal value of the control point c in the search space, and construct an adversarial bias field I for the input image bias =G bias (I; c) The neural network then outputs the original prediction p = f seg (I;θ) and the predicted value after perturbation p*=f seg (G bias (I; c); θ), calculate D by the following formula (2) comp ; where D comp represents a composite distance function used to enhance the discrimination between two predictions in segmentation; First find the optimal value of the control point c in the constructed bias field, and then optimize the parameter θ in the neural network so that the original prediction and the prediction f after generating the adversarial bias are the same. seg (G bias (I;c adv ); the distance between θ is minimized; D comp (p,p * )=D KL [p||p * ]+wD contour (p,p * ) (2) Among them, I represents the input image, θ represents the network parameters used for training, c is the control point used to construct an optimal bias field, fseg(.;θ) represents the segmentation network, Gbias(I;c) is an adversarial bias field, Dcomp represents the composite distance function, p represents the original prediction output by the neural network, p* represents the perturbed prediction output after adding the bias field, DKL represents the KL divergence, Dcontour represents a contour-based loss function, w controls the relative importance between the two equations, M represents the foreground channel, and Sx,y represents the edge extraction of two sobel filters in the x and y directions.

2. The deep learning-based fetal MRI brain tissue segmentation method according to claim 1, characterized in that: The feature pyramid model consists of a backbone network, which consists of four parts: an encoder, a decoder, a skip connection, and a feature combination; each feature map in the decoder is connected to the corresponding feature map from the encoder to form a skip connection; the skip connection connects the low-level features of the encoder with the high-level features.

3. The deep learning-based fetal MRI brain tissue segmentation method according to claim 2, characterized in that: The encoder and the decoder are configured to: introduce an attention structure CoT-Block for realizing feature extraction and utilization; the attention structure CoT-Block is used to utilize verification code key context information to guide the learning of a dynamic attention matrix.

4. The deep learning-based fetal MRI brain tissue segmentation method according to claim 3, characterized in that: The attention structure CoT-Block is configured as follows: first, a kxkxk convolution operation is used on the verification code key to encode the context information between neighbors, then the query and the key after the context information is modeled are concat, and then two consecutive 1x1x1 convolutions are used to obtain the global context information. Finally, the output result is obtained by fusing the context information between neighbors with the global context information.

5. The deep learning-based fetal MRI brain tissue segmentation method according to claim 4, characterized in that: The feature pyramid model introduces a hybrid dilated convolution module in the last layer of the decoder; the hybrid dilated convolution module increases the receptive field of the convolution kernel while keeping the number of parameters unchanged, allowing each convolution output to contain a wide range of information while ensuring that the size of the output feature map remains unchanged.

6. A fetal MRI brain tissue segmentation device based on deep learning, characterized in that: A fetal MRI brain tissue segmentation method based on deep learning as described in any one of claims 1 to 5 is used; the segmentation device comprises: A data enhancement unit, used to perform data enhancement operations on fetal brain MRI; A feature pyramid model construction unit is used to construct a feature pyramid model based on the Contextual Transformer block: the feature pyramid model introduces an attention structure CoT-Block in the encoder and decoder parts. The attention structure CoT-Block uses key context information to guide the learning of the dynamic attention matrix and enhance the features extracted from the fetal brain MRI images after data enhancement; the feature pyramid model introduces a hybrid dilated convolution module in the decoder part. The hybrid dilated convolution module expands the receptive field and retains detailed spatial information, and effectively extracts global context information in medical images; The tissue segmentation unit is used to perform tissue segmentation on the data-enhanced fetal brain MRI image using the feature pyramid model.

7. A storage medium, characterized in that: The storage medium stores a program file capable of implementing the deep learning-based fetal MRI brain tissue segmentation method described in any one of claims 1 to 5.

8. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the deep learning-based fetal MRI brain tissue segmentation method according to any one of claims 1 to 5.

Images