3D-MRI image brain tumor segmentation method integrating mixed attention and diffusion model

Abstract

The invention discloses a 3D-MRI image brain tumor segmentation method integrated with a mixed attention and diffusion model, and relates to the technical field of brain medical image analysis and processing, and the key points of the technical scheme are that brain tumor image data are inputted into a denoising diffusion probability model, and mask image data are outputted; performing feature fusion on the brain tumor image data and the mask image data, and extracting multi-scale cross-modal tumor feature data; decoding the multi-scale cross-modal tumor feature data layer by layer to obtain decoded data; the method comprises the following steps: designing a target function by fusing Kullback-Leibler (KL) divergence and Focal loss, training the whole network by using decoded data, and constructing a segmentation model; according to the method, the image brain tumor segmentation model is developed by combining the de-noising diffusion probability model and the attention mechanism, and the accuracy of an automatic segmentation algorithm and the robustness to noise are improved.

Description

Technical Field

[0001] This invention relates to the field of brain medical image analysis and processing technology, and more specifically, to a 3D-MRI image brain tumor segmentation method integrating a hybrid attention and diffusion model. Background Technology

[0002] Brain tumors, as one of the leading causes of death among infectious diseases, have garnered significant attention from healthcare professionals for their diagnosis and treatment. In clinical practice, magnetic resonance imaging (MRI) is widely used for imaging diagnosis of the brain, spinal cord, heart and major blood vessels, joints and bones, soft tissues, and pelvic cavity. Clinicians analyze the lesions of brain tumors using multi-sequence MRI imaging, such as standard sequences like FLAIR, T1, T1c, and T2. The primary focus in the differential diagnosis of brain tumors is on gliomas, the most common type of brain tumor in adults. Due to their unique anatomical structure, MRI images allow for direct examination and analysis. However, individual differences in clinical practice and the limited availability of medical resources make computer-assisted diagnosis an indispensable tool.

[0003] With the development of big data, brain tumor MRI image data samples have gradually become abundant, and supervised deep learning segmentation methods have become the mainstream segmentation algorithms. In deep learning, convolutional neural networks (CNNs) introduce semantic information when segmenting targets, injecting new vitality into semantic segmentation research. Deep convolutional neural networks, such as ResNet, Dense-Net, and FCN, have paved the way for effective analysis of computer vision tasks due to their outstanding topological structures. Fully convolutional networks (FCNs), due to their multi-scale feature extraction, encoder-decoder structure, and end-to-end approach, have become the mainstream method for current medical image processing. FCNs and their variants, 3DUNet, 3DVNet, and 3D-ResUNet, have improved the accuracy of 3D medical image segmentation, indicating that refining the network topology can effectively improve the model's segmentation ability.

[0004] Existing methods for automatic segmentation of brain tumors in 3D-MRI images based on deep neural networks mostly rely on encoder-decoder architectures for feature learning and voxel-level dense prediction, failing to adequately address issues such as blurred and uncertain tissue boundaries and noise interference. Therefore, this invention proposes a 3D-MRI image brain tumor segmentation method integrating a hybrid attention and diffusion model. Summary of the Invention

[0005] The purpose of this invention is to provide a 3D-MRI image brain tumor segmentation method that integrates a hybrid attention and diffusion model. By combining a denoising diffusion probability model and an attention mechanism to develop an image brain tumor segmentation model, the accuracy of the automatic segmentation algorithm and its robustness to noise are improved, thereby promoting the application and development of artificial intelligence in smart healthcare.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: The first aspect of this invention provides a method for segmenting brain tumors in 3D-MRI images that integrates a hybrid attention and diffusion model, comprising the following steps: Brain tumor image data is input into a denoising diffusion probability model, which outputs masked image data. Feature fusion was performed between brain tumor imaging data and masked image data to extract multi-scale, cross-modal tumor feature data; Multi-scale, cross-modal tumor feature data is decoded layer by layer to obtain the decoded data; The objective function is designed by combining Kullback-Leibler (KL) divergence and Focal loss, and the overall network is trained using the decoded data to build a segmentation model.

[0007] In conjunction with the first aspect, the present invention is further configured such that: the feature fusion method is: constructing a hybrid attention network, wherein the hybrid attention network integrates a segmentation encoder and a conditional encoder as an integrator.

[0008] In conjunction with the first aspect, the present invention is further configured to: input mask image data into a segmentation encoder and output high-frequency feature information of tumor lesions; input brain tumor image data into a conditional encoder and output multi-scale feature information of tumor tissue.

[0009] In conjunction with the first aspect, the present invention is further configured such that: the forward process of the denoising diffusion probability model progressively adds noise to the input brain tumor image data, learns its potential distribution characteristics, and then through the reverse denoising process, progressively recovers the tumor tissue mask image data with anatomical consistency from the random noise.

[0010] In conjunction with the first aspect, the present invention is further configured such that the layer-by-layer decoding is achieved through multiple sets of 3D deconvolution network layers.

[0011] In conjunction with the first aspect, the present invention is further configured such that the training method employs gradient descent.

[0012] A second aspect of the present invention also provides a 3D-MRI image brain tumor segmentation device / equipment / system integrating a hybrid attention and diffusion model, comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above method.

[0013] A third aspect of the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0014] A fourth aspect of the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0015] In summary, the present invention has the following beneficial effects: 1. The 3D-MRI brain tumor segmentation method integrating a hybrid attention and diffusion model of the present invention utilizes a denoising diffusion probability model to progressively add / denoise the labeled image, generating a mask image for auxiliary training. Compared with other methods, the method of the present invention focuses on solving the problems of uncertain, blurred, and noisy tissue boundaries in 3D-MRI brain tumor images. It combines a deep neural network to achieve parameter training for denoising and mask image generation, effectively improving the model's performance in discriminating tumor tissue boundaries.

[0016] 2. The 3D-MRI brain tumor segmentation method integrating hybrid attention and diffusion models of this invention constructs a segmentation encoder using fractional Fourier transform for the masked image generated by the diffusion model to extract high-frequency feature information of tumor lesions; for the original 3D-MRI image, a conditional encoder is constructed using hierarchical 3D convolutional network layers to extract multi-scale features of brain tissue; and a hybrid attention mechanism is designed to integrate the segmentation encoder and the conditional encoder, using offset window self-attention to calculate multi-scale cross-modal feature data of brain tumor tissue. Compared with existing methods, the method of this invention comprehensively utilizes frequency domain and spatial domain methods to deeply study the characteristics of tissue imaging data, uncover useful information hidden behind the data, and integrate it into the optimization process of the network model to jointly improve the feature learning effect of the model.

[0017] 3. The 3D-MRI brain tumor segmentation method of this invention, which integrates a hybrid attention and diffusion model, fuses KL divergence and Focal loss to design the objective function and uses gradient descent for model optimization. Compared with other methods, it can simultaneously constrain the matching between the generated distribution and the real data distribution during the diffusion process, and adaptively enhance the gradient response of difficult-to-segment lesion regions to address the severe class imbalance problem in 3D-MRI brain tumor segmentation, thereby improving the model's generation consistency and the segmentation accuracy of small-volume tumors. Attached Figure Description

[0018] Figure 1 This is a flowchart of the 3D-MRI brain tumor segmentation method integrating a hybrid attention and diffusion model in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the 3D-MRI brain tumor segmentation method integrating a hybrid attention and diffusion model in Embodiment 2 of the present invention.

[0019] Figure 3 This is the overall network architecture diagram of the 3D-MRI image brain tumor segmentation method integrating hybrid attention and diffusion models in Embodiments 1 and 2 of the present invention.

[0020] Figure 4 This is a flowchart of the forward and backward processing based on the denoising diffusion probability model of the 3D-MRI image brain tumor segmentation method integrating hybrid attention and diffusion models in Embodiments 1 and 2 of the present invention.

[0021] Figure 5 This is a structural diagram of the frequency domain channel attention mechanism of the 3D-MRI image brain tumor segmentation method integrating hybrid attention and diffusion models in Embodiments 1 and 2 of the present invention.

[0022] Figure 6 This is a structural diagram of the hybrid attention mechanism of the 3D-MRI brain tumor segmentation method integrating hybrid attention and diffusion models in Embodiments 1 and 2 of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Terminology Explanation 1. The denoising diffusion probabilistic model is a novel high-quality image generation method that generates a realistic image from noise through a progressive noise addition and inverse diffusion process. The forward process of the denoising diffusion probabilistic model adds Gaussian noise to the image, while the inverse process attempts to recreate the image from the Gaussian noise distribution.

[0025] 2. Markov chain: The probability of the current state depends only on the previous time step.

[0026] 3. Multi-scale analysis typically refers to the analysis and processing of data or signals at different spatial or temporal scales. It usually employs different filters or decomposition methods to analyze the signal structure at different scales, from low to high. Multi-scale analysis can be used to identify features at different scales, for example, detecting objects of different sizes in an image or detecting components of different frequencies in a signal.

[0027] 4. Self-attention mechanism is a mechanism that allows the model to consider the relationship between each element in the sequence and all other elements when processing the sequence. It is widely used in fields such as natural language processing and computer vision in deep learning.

[0028] 5. Frequency domain attention is a novel approach combining frequency domain analysis and attention mechanisms, designed to enhance the global modeling capabilities and feature extraction efficiency of deep learning models. By mapping input features to the frequency domain, this mechanism captures global spectral information and optimizes the weight allocation of key features through attention, thereby improving model performance and robustness. The core of the frequency domain attention mechanism lies in using Fourier transform to convert input features from the spatial domain to the frequency domain, extracting global spectral features. Subsequently, the frequency domain features are weighted using the attention mechanism to enhance the responsiveness to key regions.

[0029] Example 1: A method for segmenting brain tumors in 3D-MRI images integrating a hybrid attention and diffusion model, such as Figure 1 As shown, it includes the following steps: S1: Generation of 3D mask images of brain tumors based on a denoising diffusion probability model: High-precision mask image data is synthesized from random noise through a stepwise denoising generation process.

[0030] S2: Multiscale cross-modal brain tumor feature encoding based on hybrid attention mechanism: By designing a hybrid mechanism based on frequency domain channel attention and cross self-attention, the original 3D-MRI image data and the masked image data generated in step S1 are fused to extract multiscale cross-modal brain tumor feature data.

[0031] S3: Model training by fusing Kullback-Leibler (KL) divergence and Focal loss: Based on the multi-scale cross-modal brain tumor feature data extracted in step S2, the model is decoded layer by layer, and the objective function is designed by fusing KL divergence and Focal loss. The model is then trained using the gradient descent method.

[0032] In this embodiment, the specific process of step S1 is as follows: S1.1: The input 3D-MRI brain tumor image data is progressively denoised using the forward process of the denoising diffusion probability model to learn its potential distribution characteristics.

[0033] S1.2 uses a reverse denoising process to gradually recover an anatomically consistent tumor tissue mask image from random noise.

[0034] In this embodiment, the specific process of step S1.1 is as follows: S1.1.1: Forward process based on Gaussian noise erosion: By progressively altering the original input mask image... Gaussian noise is added for erosion, resulting in a final image with only noise. Suppose that at a certain time step in the forward process... images Depends on previous time steps images This process can be viewed as a Markov chain that satisfies: (1) in, Indicates the forward process, Indicates a normal distribution. It is the identity matrix. It is used for the current time step Control the variance of the noise level and satisfy the following conditions: Using formula S1, sample image data. Generate random variables that follow a standard normal distribution. , making satisfy (2) in ,and .

[0035] In this embodiment, the specific process of step S1.2 is as follows: S1.2.1: Image data generated at each time step based on step S1.1.1 Training deep neural networks to fit the distribution This performs the reverse denoising process, and the calculation model is as follows: (3) in, This indicates the deep learning model. These are the parameters of the depth model. This represents the mean of the model parameters. This represents the variance of the model parameters. It is the noise reduction time step. Represents a normal distribution. The initial mask image of the input is known. The distribution that the neural network needs to fit It can be calculated using Bayes' theorem: (4) in, and These represent the results after the reverse process is executed. Mean and variance of time steps.

[0036] S1.2.2 Based on the calculation model in step S1.2.1, the inverse process of the denoising diffusion model is transformed into the forward process using Bayes' theorem, defined by the Gaussian probability density function. Right now (5) (6) According to formula (5) in S1.1.1, Calculated by the following formula (7) Substituting this into formula (6), we get (8) As can be seen from the above steps, a deep neural network model is needed to approximate the distribution. According to formula (6), the model is trained. Using the estimation formula (8). When Known during the training phase and used as input, the model estimates the noise using the following formula. : (9) According to formula (9), the final result is (10) S2. Multiscale cross-modal brain tumor feature encoding based on hybrid attention mechanism: By designing a hybrid mechanism based on frequency domain channel attention and cross self-attention, the original 3D-MRI image data and the masked image generated in step S1 are fused to extract multiscale cross-modal brain tumor features.

[0037] In this embodiment, the specific process of step S2 is as follows: S2.1: For raw 3D-MRI brain tumor image data, a hierarchical 3D convolutional network layer is designed to extract multi-scale features of brain tissue and construct a conditional encoder.

[0038] S2.2: For the generated masked image data, a fractional Fourier transform layer is designed to extract multi-channel high-frequency features of tumor lesions, and a segmentation encoder is constructed.

[0039] S2.3: For the multi-scale features of brain tissue output in step S2.1 and the multi-channel high-frequency features of tumor lesions output in step S2.2, an offset window multi-head self-attention mechanism is designed to fuse them, thereby realizing multi-scale cross-modal brain tumor feature encoding.

[0040] In this embodiment, the specific process of step S2.1 is as follows: (2.1.1) For the input source 3D-MRI image The conditional encoder, consisting of four sets of convolutional computation modules, extracts multi-scale features from the image data. Each convolutional module comprises a 3D convolutional layer, a normalization layer, and an activation layer, defined as follows: (16) in Indicates size is 3D convolution kernels, Presentation layer normalization, This represents the Parametric Rectified Linear Unit activation function. .

[0041] The specific process of step (2.2) is as follows: (2.2.1) To extract multi-channel frequency feature data from source 3D-MRI image data and obtain detailed information with varying levels of richness, a frequency channel attention mechanism based on fractional Fourier convolution is designed. Given a spatial feature map... ( (The coordinates of a feature point in the spatial domain), the feature calculated by discrete fractional Fourier convolution in the frequency domain. ( (The coordinates of the feature points in the frequency domain) are obtained by element-wise multiplication of the spatial feature map with the convolution kernel, and are defined as: (11) in, It indicates the fractional order. They represent along Fractional Fourier convolution kernel functions in three directions.

[0042] (2.2.2) Based on the frequency domain feature map obtained in step (2.2.1), global max pooling is used. Global average pooling The Sigmoid function calculates the attention coefficient for each channel of the feature map, and the specific calculation model is as follows: (12) To focus on high-frequency components with lower grayscale values ​​in the frequency domain, the following activation function is used to enhance frequency domain channel attention: Finally, the frequency domain feature map is multiplied element-wise with the attention. Obtain attention-enhanced frequency domain feature maps: (13) (2.2.3) The frequency domain feature mapping obtained from step (2.2.2) The inverse fractional Fourier transform is used to obtain the contrast-enhanced feature map, defined as follows: (14) They represent along 3D inverse fractional Fourier convolution kernel function in three directions.

[0043] In this embodiment, the specific process of step (2.3) is as follows: (2.3.1) For the feature map output by the conditional encoder in step (2.1) The feature map output by the segmented encoder in step (2.2) We design a trainable linear transform network layer to encode spatial information from the input data and obtain the location embedding. The specific calculation is as follows: (15) in Indicates size is 3D convolution kernels, This represents a spatial information encoding function for a linear transformation.

[0044] (2.3.2) Based on the position embedding obtained in step (2.3.1), the feature data output by the segment encoder and the condition encoder are linearly transformed to form a query, key, and value matrix: (16) in This means copying the input matrix data three times. Then, three sets of convolutional kernels with different sizes are applied to each copy. 3D convolution Perform a linear transformation on these matrices to obtain the query ( ),key( ) and value ( )matrix.

[0045] (2.3.3) Based on the query, key, and value matrix obtained in step (2.3.2), the query matrix generated by the conditional encoder is applied to the key matrix of the segmentation encoder to calculate the similarity between the two and obtain the self-attention score. This score quantifies the degree of attention paid by the conditional encoder to the features of the segmentation encoder at different positions. The single attention calculation model is calculated by the Softmax function as follows: (17) in This represents the query matrix of the condition encoder. This represents the key matrix of the segment encoder. This represents the value matrix of the segmented encoder. This represents the dimensions of the three matrices. These represent the learnable weight matrices.

[0046] (2.3.4) Calculate a single attention head according to step (2.3.3). The entire cross-self-attention mechanism is constructed from 4 groups of multi-head attention modules. The calculation of each group is as follows: (18) in , These represent the multi-head attention calculations performed by partitioning the regular window and the offset window, respectively. This represents a multilayer perceptron. Presentation layer normalization.

[0047] S3. Model training by fusing Kullback-Leibler (KL) divergence and Focal loss: Based on the multi-scale cross-modal brain tumor feature data extracted in step S2, hierarchical decoding is performed, and the objective function is designed by fusing KL divergence and Focal loss. The model is then trained using the gradient descent method.

[0048] In this embodiment, the specific process of step S3 is as follows: (3.1) Design multiple 3D deconvolution network layers to achieve layer-by-layer feature decoding.

[0049] (3.2) The objective function is designed by combining KL divergence and Focal loss, and the model is trained by gradient descent.

[0050] In this embodiment, the specific process of step (3.1) is as follows: (3.1.1) For the feature data output by the hybrid attention encoder in step (2.3.4), a decoder based on 3D deconvolution is designed. The encoder consists of 4 sets of deconvolution calculation modules, and each deconvolution calculation module is defined as follows: (19) in Indicates size is 3D deconvolution kernel, This indicates a feature concatenation operation. .

[0051] In this embodiment, the specific process of step (3.2) is as follows: (3.2.1) Based on the feature data obtained from the deconvolution calculation in step (3.1.1), given the image mask pair and the Image generated by time-step diffusion Based on the variational upper bound of the negative log-likelihood of the diffusion probability model, the objective function for training this network model is designed by integrating the Kullback-Leibler (KL) divergence and the Focal loss function. The specific calculation model is as follows: in These represent the true class label and the estimated Gaussian noise, respectively. Used to balance the relative errors of correct and incorrect classifications. This represents the mathematical expectation. Ultimately, the objective function of the entire network is defined as... The regularization coefficient is used to adjust the weights of the Focal loss. Finally, gradient descent is used to train the neural network and build a segmentation model.

[0052] Example 2: A method for segmenting brain tumors in 3D-MRI images integrating a hybrid attention and diffusion model, such as Figure 2 The process includes the following steps: S100. Input the brain tumor image data into the denoising diffusion probability model and output the masked image data; S200. Feature fusion is performed between brain tumor imaging data and masked image data to extract multi-scale, cross-modal tumor feature data; S300. Decode the multi-scale, cross-modal tumor feature data layer by layer to obtain the decoded data; S400. The objective function is designed by fusing Kullback-Leibler (KL) divergence and Focal loss. The decoded data is used to train the overall network to build a segmentation model.

[0053] In this embodiment, the feature fusion method is as follows: a hybrid attention network is constructed, which integrates a segmentation encoder and a conditional encoder. Masked image data is input into the segmentation encoder, which outputs high-frequency feature information of the tumor lesion; brain tumor image data is input into the conditional encoder, which outputs multi-scale feature information of the brain tissue.

[0054] In this embodiment, the forward process of the denoising diffusion probability model progressively adds noise to the input brain tumor image data to learn its potential distribution characteristics. Then, through the reverse denoising process, the brain tissue mask image data with anatomical consistency is gradually recovered from the random noise.

[0055] In this embodiment, layer-by-layer decoding is achieved through multiple sets of 3D deconvolutional network layers.

[0056] In this embodiment, the training method uses gradient descent.

[0057] The present invention also provides an apparatus / device / system for a 3D-MRI brain tumor segmentation method integrating a hybrid attention and diffusion model, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0058] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0059] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0060] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for segmenting brain tumors in 3D-MRI images integrating a hybrid attention and diffusion model, characterized by: Includes the following steps: Brain tumor image data is input into a denoising diffusion probability model, which outputs masked image data. Feature fusion was performed between brain tumor imaging data and masked image data to extract multi-scale, cross-modal tumor feature data; Multi-scale, cross-modal tumor feature data is decoded layer by layer to obtain the decoded data; The objective function is designed by combining Kullback-Leibler (KL) divergence and Focal loss. The entire network is trained using the decoded data to construct a well-trained segmentation model.

2. The intelligent segmentation method according to claim 1, characterized in that: The feature fusion method is as follows: construct a hybrid attention network, which integrates a segmentation encoder and a conditional encoder.

3. The intelligent segmentation method according to claim 2, characterized in that: The masked image data is input into the segmentation encoder, which outputs high-frequency feature information of the tumor lesion; the brain tumor image data is input into the conditional encoder, which outputs multi-scale feature information of the tumor tissue.

4. The intelligent segmentation method according to claim 1, characterized in that: The denoising diffusion probability model's forward process progressively adds noise to the input brain tumor image data to learn its potential distribution characteristics. Then, through a reverse denoising process, tumor tissue mask image data with anatomical consistency is gradually recovered from random noise.

5. The intelligent segmentation method according to claim 1, characterized in that: The layer-by-layer decoding is achieved through multiple sets of 3D deconvolutional network layers.

6. The intelligent segmentation method according to claim 1, characterized in that: The training method used is gradient descent.

7. A 3D-MRI brain tumor segmentation device / equipment / system integrating a hybrid attention and diffusion model, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-6.

Images