Medical image segmentation method and system based on RU-Net model

Abstract

The invention provides a medical image segmentation method and system based on an RU-Net model. The method comprises the following steps: S1, inputting an image into the RU-Net model; s2, capturing basic features in the image through a series of first convolutional layers in the encoder; s3, reducing the spatial dimension of the features through a pooling layer, and extracting abstract features; s4, the features are processed through a recursive convolutional layer, so that the model can accumulate and learn the features on a plurality of time steps, and the understanding of deep features of the image is enhanced; s5, recovering the spatial resolution of the feature map through an up-sampling layer in the decoder, and fusing the spatial resolution with the feature map from the encoder; s6, further processing the fused feature map through a series of second convolutional layers and RCL, and reconstructing a fine structure of the image; and S7, outputting a segmentation result through a final convolutional layer. According to the method, the abnormal areas in the medical image can be automatically detected and accurately classified, so that the diagnosis efficiency and accuracy are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image analysis, in particular, to a medical image segmentation method and system based on a RU-Net model. BACKGROUND

[0002] In the field of medical image segmentation, traditional segmentation methods mainly rely on manually drawn regions and predefined features. These methods not only consume a lot of time and effort, but also have a high dependence on expert experience and prior knowledge, making it difficult to meet the accuracy and efficiency requirements of modern medicine when dealing with complex and variable medical images.

[0003] With the explosive growth of medical imaging data, traditional segmentation methods have been unable to meet the rapid, automated and high-precision clinical needs. Therefore, a new solution combining deep learning technology with medical image segmentation has emerged. Currently, there are a variety of deep learning-based medical image segmentation frameworks, such as U-Net-based segmentation networks, V-Net-based segmentation networks, etc. These methods have significantly improved in terms of automation and segmentation accuracy compared to traditional methods.

[0004] For example, Chinese patent application publication CN108596915A discloses a medical image segmentation method based on unlabeled data, including the following steps: A1: training the segmentation network using labeled data; A2: inputting training images containing unlabeled data and labeled data into the segmentation network for segmentation, and inputting the obtained segmentation results into the evaluation network; A3: evaluating the segmentation results using the evaluation network; A4: performing adversarial training on the segmentation network and the evaluation network; A5: when the segmentation network and the evaluation network reach a dynamic balance, obtaining the final segmentation results of the unlabeled data based on the segmentation network.

[0005] For another example, Chinese patent application publication CN118485643A discloses a medical image analysis processing system based on image analysis, including an image acquisition module, an image segmentation module, a feature extraction module, a decision analysis module and a result output module, wherein the image segmentation module is used to segment lesion region images and organ region images from preprocessed medical image data through a fusion model, and the fusion model is specifically a multi-scale U-Net, a PSTNet, a hybrid CNN-Transformer and a multi-task learning fusion model.

[0006] These methods still face challenges in real-time and accuracy when dealing with highly complex and variable medical images, especially in terms of small sample learning, class imbalance and model generalization ability. There is still much room for improvement in existing learning models. SUMMARY

[0007] In order to solve the above problems, the application provides a medical image segmentation method and system based on an RU-Net model, which can automatically detect abnormal regions in a medical image and accurately classify them, thereby greatly improving the efficiency and accuracy of diagnosis and providing a powerful auxiliary tool for clinicians.

[0008] The application provides a medical image segmentation method based on an RU-Net model, comprising the following steps: S1, inputting an image into an RU-Net model; S2, capturing basic features in the image through a series of first convolutional layers in an encoder; S3, reducing the spatial dimension of the features through a pooling layer to extract abstract features; S4, processing the features through a recurrent convolutional layer to enable the model to accumulate and learn the features at multiple time steps, thereby enhancing the understanding of deep features of the image; S5, restoring the spatial resolution of the feature map through an up-sampling layer in the decoder and fusing the feature map with the feature map from the encoder; S6, further processing the fused feature map through a series of second convolutional layers and RCL to reconstruct the fine structure of the image; S7, outputting a segmentation result through a final convolutional layer.

[0009] Preferably, the step S2 further comprises applying an activation function to increase the nonlinearity of the model processing.

[0010] Preferably, the training process of the RU-Net model comprises: After the MRI brain image data processed by standardization and size unification, the MRI brain image data is divided into a training set, a validation set and a test set according to a proportion, and the data diversity is expanded by using a data augmentation technique; during training, batch data is input into the model through a data loader, and each batch contains multiple 256x256 pixel images and their corresponding labels; In the model forward propagation stage, the recurrent convolutional layer in the encoder merges the current convolutional output with the original image, while accumulating the feature output at the previous time, and continuously captures the detailed information of the image; The decoder also uses the recurrent convolutional layer and the residual connection to accurately map the abstract information transmitted by the encoder to the pixel-level segmentation result in the process of restoring the resolution of the feature map.

[0011] Preferably, the training of the RU-Net model adopts a multi-loss function fusion strategy, including a cross-entropy loss function and a Dice loss function. The cross-entropy loss function measures the difference between the predicted category and the real label, and optimizes the pixel classification accuracy; the Dice loss function focuses on the integrity of the segmentation region, and solves the class imbalance problem of the tumor region with a small proportion in the medical image.

[0012] The application further provides a medical image segmentation system based on the RU-Net model, comprising: An image input module is configured to input an image into the RU-Net model. A basic feature capturing module is configured to capture basic features in the image through a series of first convolutional layers in an encoder. An abstract feature extraction module is configured to reduce the spatial dimension of the features through a pooling layer and extract abstract features. A cumulative and learning feature module is configured to process the features through a recursive convolutional layer, so that the model can accumulate and learn the features at multiple time steps, thereby enhancing the understanding of deep features of the image. A feature map fusion module is configured to restore the spatial resolution of the feature map through an up-sampling layer in the decoder and fuse the feature map with the feature map from the encoder. A reconstructed image module is configured to further process the fused feature map through a series of second convolutional layers and RCL, and reconstruct the fine structure of the image. A segmentation result output module is configured to output the segmentation result through a final convolutional layer.

[0013] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the medical image segmentation method when executing the computer program.

[0014] The application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the medical image segmentation method.

[0015] The application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the steps of the medical image segmentation method.

[0016] Advantages

[0017] The application has the following advantages: the medical image segmentation method and system based on the RU-Net model can automatically detect abnormal regions in the medical image and accurately classify them even when processing medical images with high complexity and variability, thereby greatly improving the efficiency and accuracy of diagnosis and providing a powerful auxiliary tool for clinicians. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of a medical image segmentation method based on a RU-Net model according to an embodiment of the present application; Figure 2 is a structural block diagram of a RU-Net model used according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Embodiments of the present application will be described below with reference to the accompanying drawings, and those skilled in the art will understand that these embodiments are only illustrative, not limiting.

[0020] Figure 1 is a flowchart of a medical image segmentation method based on a RU-Net model according to an embodiment of the present application. As shown in Figure 1 , the medical image segmentation method based on the RU-Net model according to the embodiment of the present application includes the following steps: S1, inputting an image into a RU-Net model; S2, capturing basic features in the image through a series of first convolutional layers in an encoder; the encoder includes two 3x3 convolutional layers and a ReLU activation function; S3, reducing the spatial dimension of the features through a pooling layer to extract abstract features; S4, processing the features through a recursive convolutional layer so that the model can accumulate and learn the features over multiple time steps, thereby enhancing the understanding of deep features of the image; S5, restoring the spatial resolution of the feature map through an up-sampling layer in the decoder and fusing it with the feature map from the encoder; S6, further processing the fused feature map through a series of second convolutional layers and RCL to reconstruct the fine structure of the image; S7, outputting the segmentation result through a final convolutional layer.

[0021] Taking lung CT tumor segmentation as an example, the above steps are specifically described as follows: In the lung CT tumor segmentation task, the running process of the RU-Net model is as follows. First, a single-channel grayscale CT image with a resolution of 256x256 is input into the model, and the image needs to be normalized before inputting to scale the pixel value to the [0, 1] interval to improve the training efficiency of the model. Then, the image enters the encoder, and the basic features such as edges and textures are extracted through the module composed of two 3x3 convolutional layers and a ReLU activation function, at which time the channel number of the feature map is expanded from 1 to 64, and the size remains unchanged. Subsequently, a 2x2 max pooling layer is used to halve the resolution of the feature map, while reducing the amount of calculation and extracting abstract features such as the approximate shape of the tumor.

[0022] To further explore the image features, the model uses recursive convolution layers (RCL) to process abstract features. Taking three recursions as an example, each recursion will perform a convolution operation on the input features and accumulate the output results, enabling the model to learn features over multiple "time steps" and effectively distinguish the texture differences within the tumor. In the decoder stage, the feature map resolution is restored to the original size through transposed convolution, and the feature map is spliced with the feature map of the corresponding layer of the encoder to fuse abstract semantic information and detailed features. The spliced feature map is further refined through a second group of convolution layers and RCL, further refining the tumor boundary, such as reconstructing the fine structure of the burr-shaped edge. Finally, the channel number is compressed to 1 using a 1x1 convolution, and the segmentation probability mask is output using a Sigmoid activation function, with the region where the probability value is greater than 0.5 being determined as the tumor, completing the entire segmentation process.

[0023] The entire process of the medical image segmentation method based on the RU-Net model of the embodiment of the present application not only shows how the RU-Net model handles the image segmentation task, but also reveals how the encoder and the decoder work together to balance feature extraction and image reconstruction. This design enables the RU-Net model to achieve high precision and robustness when processing complex medical image segmentation tasks, especially when dealing with images with high noise and variability.

[0024] Optionally, after step S2, an activation function is applied to increase the nonlinearity of the model processing.

[0025] In the lung CT tumor segmentation task, after the image is input into the RU-Net model, step S2 captures the basic features through a series of first convolution layers in the encoder. Taking the first convolution operation as an example, 64 3x3 convolution kernels are used, and during the convolution process, the feature map space size is maintained by setting padding=1. At this time, the input 256x256x1 lung CT grayscale image is output as a 256x256x64 feature map after convolution, and edge and texture information such as lung vessel contours are preliminarily extracted.

[0026] However, the convolution operation at this time is essentially a linear transformation, which is difficult to learn the complex nonlinear relationships in the data. To solve this problem, an activation function is applied after each convolution operation, the most commonly used being the ReLU (Rectified Linear Unit) function. Its mathematical expression is f(x)=max(0,x), that is, when the input value is less than 0, the output is 0; when the input value is greater than 0, the output is equal to the input value. In processing lung CT image features, the ReLU function "filters" out invalid information (such as negative pixel values caused by noise) in the feature map that is less than 0, and retains and enhances the positive effective feature signals, such as highlighting the positive gradient information of the tumor edge.

[0027] Figure 2 is a structural block diagram of the RU-Net model used in the present application. The RU-Net model comprises an image input layer (Input Image), a recursive convolution layer (RCL), a pooling layer (Pooling), a feature extraction layer (Feature), an up-sampling layer (Upsampling), and an output layer (Output).

[0028] The RU-Net (Residual U-Net) model is a deep learning model based on the U-Net architecture and incorporating the residual learning concept, widely used in medical image segmentation and other fields. It adopts an encoder-decoder structure, with the encoder composed of convolutional layers and pooling layers, responsible for encoding the image input into the RU-Net model, extracting semantic information from low to high levels, and reducing the resolution of the feature map through the pooling layer to expand the receptive field. The decoder is composed of up-sampling layers and convolutional layers, and its task is to restore the low-resolution feature map extracted by the encoder to a resolution close to the image input into the RU-Net model, and generate the final segmentation result. In RU-Net, the encoder and decoder work closely together, the encoder provides basic features for the decoder, the decoder performs segmentation based on these features and feeds back optimization information to the encoder, through the residual learning mechanism, both learn and optimize each other, and together improve the performance of the model in complex image segmentation tasks, enabling the model to more accurately restore image details and segment target regions.

[0029] RU-Net introduces recursive convolution layers (RCLs), enabling the network to learn features over multiple time steps, thus achieving deep accumulation and richer contextual understanding of image features. This dynamic feature learning mechanism effectively improves the model's performance in complex medical images, especially when dealing with images with high noise and variability, showing stronger robustness.

[0030] The present application also provides a medical image segmentation system based on the RU-Net model, comprising: an image input module for inputting an image into the RU-Net model; a basic feature capturing module for capturing basic features in the image through a series of first convolutional layers in the encoder; an abstract feature extraction module for reducing the spatial dimension of the features through the pooling layer to extract abstract features; a cumulative and learning feature module for processing the features through the recursive convolution layer, enabling the model to accumulate and learn the features over multiple time steps, thus enhancing the understanding of deep image features; a feature map fusion module for restoring the spatial resolution of the feature map through the up-sampling layer in the decoder and fusing it with the feature map from the encoder; The reconstructed image module is configured to further process the fused feature map through a series of second convolutional layers and RCLs to reconstruct fine structures of the image. The segmentation result output module is configured to output the segmentation result through a final convolutional layer.

[0031] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the medical image segmentation method when executing the computer program.

[0032] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the medical image segmentation method when executed by a processor.

[0033] The application further provides a computer program product comprising a computer program, wherein the computer program implements the steps of the medical image segmentation method when executed by a processor.

[0034] Embodiments

[0035] In application to a specific example, a medical image segmentation method based on a RU-Net model comprises the following steps: (1) Data preparation MRI brain images in a public medical image database are selected as data sources, and the images contain labeled tumor regions and other normal tissue regions. The images are standardized, the pixel values are normalized to the range of [0, 1], and the image size is uniformly cropped and scaled to 256x256 pixels to meet the input requirements of the model.

[0036] (2) Model construction Encoder: The encoder is composed of multiple convolutional blocks, each of which contains two convolutional layers and a max-pooling layer. In RU-Net, some modules are replaced with recurrent convolutional layers (RCLs), and the output of this module can be represented as. The so-called residual is to combine the convolutional output and the original image, and the recurrence not only accepts the combined output of the current time and the previous time. This structure helps to accumulate features during encoding, providing richer feature representations for subsequent segmentation tasks.

[0037] Decoder: The decoder is composed of up-sampling layers and convolutional layers, and its task is to restore the low-resolution feature maps extracted by the encoder to a resolution close to the input image and generate the final segmentation result. During the decoding process, recurrent convolutional layers and residual connections are also used to further optimize feature restoration and segmentation accuracy.

[0038] (3) Model training and optimization During the training process, RU-Net can better learn the details and structural information of the image through residual learning and feature accumulation of the recurrent convolutional layer.

[0039] In the medical image segmentation task, the training process of the RU-Net model takes optimizing the segmentation accuracy as the core goal, and fully utilizes the advantages of residual learning and recurrent convolutional layers. First, the MRI brain image data after standardization and size unification processing is divided into training set, validation set and test set according to the proportion, and the data diversity is expanded by using data enhancement techniques such as random rotation, brightness adjustment, noise addition, etc. to avoid model overfitting. During training, batch data is input into the model through the data loader, and each batch contains multiple 256x256 pixel images and their corresponding annotations.

[0040] During the forward propagation of the model, the recurrent convolutional layer (RCL) in the encoder plays a key role. It not only merges the current convolution output with the original image (residual connection), but also accumulates the feature output of the previous time to continuously capture details such as tumor boundaries and textures in the image. For example, when identifying the core area of brain tumors, RCLs gradually strengthen the extraction of irregular edge features specific to tumors through multiple cycles. The decoder part also uses recurrent convolutional layers and residual connections to further optimize the detail features during the process of restoring the resolution of the feature map, and accurately map the abstract information passed by the encoder to the pixel-level segmentation result.

[0041] In order to guide the model learning, the training adopts the strategy of multi-loss function fusion. The cross-entropy loss function measures the difference between the predicted category and the true label, focusing on optimizing the pixel classification accuracy; the Dice loss function focuses on the integrity of the segmentation region, effectively solving the class imbalance problem of small tumor area in medical images. Combine the two with certain weights to form the total loss function, guide the model to improve the classification accuracy and regional segmentation effect at the same time. During the training process, the Adam optimizer is used to minimize the total loss, and the model parameters are updated through the back propagation algorithm, and the learning rate decay strategy is used at the same time, such as reducing the learning rate by half when the validation set loss does not decrease for 5 consecutive epochs, to ensure that the model can still converge to a better solution in the later training period. After each round of training, the model performance is evaluated using the validation set, the loss value and segmentation indicators (such as Dice coefficient) are monitored, and the hyperparameters are adjusted in time until the model reaches the best performance on the validation set. Finally, the generalization ability of the trained model is evaluated using the test set.

[0042] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A medical image segmentation method based on the RU-Net model, characterized by: The following steps are involved: S1. Input the image into the RU-Net model; S2, through a series of first convolutional layers in the encoder, captures the basic features in the image; S3, through the pooling layer to reduce the spatial dimension of the feature and extract abstract features; S4, processing features through recursive convolutional layers, enables the model to accumulate and learn features over multiple time steps, thereby enhancing the understanding of deep image features; S5, restore the spatial resolution of the feature map through the upsampling layer in the decoder and fuse it with the feature map from the encoder; S6, the fused feature map is further processed through a series of second convolutional layers and RCL to reconstruct the fine structure of the image; S7, output the segmentation result through a final convolutional layer.

2. The medical image segmentation method according to claim 1, wherein: After step S2, the method further includes applying an activation function to increase the nonlinearity of the model processing.

3. The medical image segmentation method according to claim 1, wherein: The training process of the RU-Net model includes: The standardized and resized MRI brain image data was divided into training, validation, and test sets according to their proportions, and data augmentation techniques were used to increase data diversity. During training, batches of data were fed into the model through a data loader. Each batch contained multiple 256×256 pixel images and their corresponding annotations. In the forward propagation phase of the model, the current convolution output of the recurrent convolution layer in the encoder is merged with the original image, while accumulating the feature output of the previous moment to continuously capture the detailed information of the image; The decoder also uses recurrent convolutional layers and residual connections to accurately map the abstract information passed by the encoder into pixel-level segmentation results while restoring the resolution of the feature map.

4. The medical image segmentation method according to claim 3, wherein: The training of the RU-Net model adopts a multi-loss function fusion strategy, including the cross entropy loss function and the Dice loss function; The cross entropy loss function measures the difference between the predicted category and the true label, optimizing the pixel classification accuracy; the Dice loss function focuses on the integrity of the segmented area, solving the class imbalance problem in medical images where the tumor area accounts for a small proportion.

5. A medical image segmentation system based on the RU-Net model, comprising: Image input module, used to input images into the RU-Net model; A basic feature capture module, which is used to capture the basic features of the image through a series of first convolutional layers in the encoder; Abstract feature extraction module, used to reduce the spatial dimension of features through pooling layers and extract abstract features; The accumulation and learning feature module is used to process features through recursive convolutional layers, enabling the model to accumulate and learn features over multiple time steps, thereby enhancing the understanding of deep image features; A feature map fusion module is used to restore the spatial resolution of the feature map through the upsampling layer in the decoder and fuse it with the feature map from the encoder; The image reconstruction module is used to further process the fused feature map through a series of second convolutional layers and RCL to reconstruct the fine structure of the image; The segmentation result output module is used to output the segmentation result through a final convolutional layer.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the medical image segmentation method according to claim 1 or 2 when executing the computer program.

7. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the medical image segmentation method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the medical image segmentation method according to any one of claims 1 to 4 are implemented.

Images