A Medical Image Segmentation Method Based on Graph Neural Networks and U-Net Deep Learning Framework

Abstract

This invention discloses a medical image segmentation method based on graph neural networks and the U-Net deep learning framework, belonging to the field of smart healthcare technology. It provides a fully automated end-to-end solution for medical image segmentation, capable of handling images of different formats and resolutions generated by existing imaging systems. The system first normalizes the input image, extracts patches or scales them, and processes them to the network input size. The normalized image is then segmented using a pre-trained VGU-Net model to segment the target lesion region. Compared to existing methods, this invention achieves fully automated segmentation and significantly improves segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the field of smart medical technology, and in particular to a medical image segmentation method based on graph neural networks and the U-Net deep learning framework. Background Technology

[0002] Medical image segmentation divides an image into non-overlapping regions, each corresponding to a specific meaning. This method is helpful for detecting abnormal areas such as tumors, polyps, and stones, segmenting specific organs, and for assisted robotic surgery in localization and precise cutting. Its importance lies in its ability to significantly improve image processing speed, help doctors better identify lesion areas, and improve the accuracy of medical diagnoses.

[0003] Deep learning technology is very effective in developing image segmentation systems, especially in situations with limited domains. Deep learning can achieve good generalization ability even with small datasets.

[0004] U-Net is one of the representative convolutional neural networks. It is arguably the most commonly used and simplest segmentation model. It is simple, efficient, easy to understand, easy to build, and can be trained on small datasets. U-Net was proposed in 2015 in the paper "U-Net: Convolutional Networks for Biomedical Image Segmentation". U-Net was initially designed to solve the problem of medical image segmentation. In the task of segmenting at the cell level, it won multiple first places in the 2015 ISBI celltracking competition. Subsequently, U-Net has been widely applied in various directions of semantic segmentation due to its outstanding segmentation performance.

[0005] However, such convolutional neural networks are usually limited by the local feature extraction capability of the convolutional kernel, making it difficult to interact with features that are far apart in the image region. This may result in insufficient feature representation capability extracted by the deep learning model, leading to poor performance in downstream tasks such as classification or segmentation.

[0006] Therefore, those skilled in the art are dedicated to developing a medical image segmentation method based on graph neural networks and the U-Net deep learning framework. A novel deep learning model, VGU-Net, has been designed, providing end-to-end fully automated medical image segmentation software. It performs image homogenization and size compression processing, and uses the VGU-Net contour model to segment regions of interest in regularized images. Summary of the Invention

[0007] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is that convolutional neural networks are limited by their ability to extract local features, resulting in poor performance in medical image classification or segmentation.

[0008] To achieve the above objectives, this invention provides a medical image segmentation method based on graph neural networks and the U-Net deep learning framework, comprising the following steps:

[0009] Step 1: Input image;

[0010] Step 2: Image regularization processing;

[0011] Step 3: Obtain a training set of regularized images in the target image domain, train and optimize the VGU-Net model, and obtain the model that performs best on the validation set.

[0012] Step 4: Use the trained VGU-Net model to predict the segmentation result of the input image.

[0013] Furthermore, the input image in step 1 includes JPG format, PNG format, and DCM format.

[0014] Furthermore, step 2 includes image homogenization and size scaling processing.

[0015] Furthermore, step 2 includes performing a pixel value inversion operation on the DCM format image.

[0016] Furthermore, step 2 includes performing bilinear interpolation scaling on the image.

[0017] Furthermore, in step 3, the VGU-Net model integrates convolutional neural networks and graph neural networks.

[0018] Furthermore, the convolutional neural network adopts the U-Net network framework.

[0019] Furthermore, the graph neural network employs a graph convolutional neural network.

[0020] Furthermore, the convolutional neural network extracts image features.

[0021] Furthermore, the graph neural network interacts with image features to transmit information.

[0022] In a preferred embodiment of the present invention, addressing the limitation of convolutional neural networks in local feature extraction, a novel VGU-Net medical image segmentation model is proposed, combining graph neural networks with convolutional neural networks. Convolutional neural networks are used to extract local features, and graph neural networks are used to facilitate information interaction between these local features. This achieves a fully automated end-to-end image segmentation model with significantly improved segmentation accuracy.

[0023] Compared with the prior art, the present invention has the following obvious substantive features and significant advantages:

[0024] 1. Achieve a fully automated end-to-end medical image segmentation workflow.

[0025] 2. The VGU-Net model significantly improves the accuracy of medical image segmentation compared to the most mature existing models.

[0026] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0027] Figure 1 This is a VGU-Net network structure diagram of a preferred embodiment of the present invention;

[0028] Figure 2 This is a flowchart of a preferred embodiment of the present invention;

[0029] Figure 3 This is a flowchart illustrating a preferred embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the input image of a preferred embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of a regularized image according to a preferred embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram showing a preferred embodiment of the present invention. Detailed Implementation

[0033] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0034] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0035] This invention designs a novel deep learning model, VGU-Net, and provides end-to-end fully automated medical image segmentation software. It performs image homogenization and size compression processing, and uses the VGU-Net contour model to segment regions of interest in regularized images.

[0036] VGU-Net segmentation model: VGU-Net is a model that combines the advantages of Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs). The CNN primarily uses the U-Net network framework, while the GNN employs the commonly used Graph Convolutional Neural Network (GCN).

[0037] The VGU-Net network structure is as follows: Figure 1 As shown, its network structure is symmetrical and largely similar to U-Net. The convolutional part is responsible for extracting local information from the image. After each downsampling, a 2x2 patch from the previous-level feature map is constructed as a node in the graph. The features corresponding to the patch are flattened into vectors as the node's features. If the length and width dimensions of the previous-level feature map F are H and W respectively, then the total number of constructed graph nodes is... The connectivity between nodes in a graph is represented by an adjacency matrix. The self-attention mechanism of the feature graph is used to calculate the similarity between nodes, which is then used as the weight of the edges.

[0038] The main advantages of VGU-Net are:

[0039] 1. Convolutional neural networks and graph neural networks each have their own significance: Convolutional neural networks can extract image features; graph neural networks can enable interaction between features and transmit information.

[0040] 2. Through continuous downsampling, the larger the scale of image information corresponding to a graph node, the richer the information it contains. Multi-layer downsampling enables graph nodes to contain information at different scales.

[0041] Since input images may come in various formats and resolutions, regularization is the first step. Because medical image segmentation software operates in an end-to-end manner, the processed, regularized image is input into the VGU-Net model to obtain the segmentation visualization results.

[0042] A diagram of the workflow is shown below. Figure 2 As shown: the preprocessing (regularization) part includes steps 1 and 2, and the subsequent segmentation and extraction of the region of interest part includes steps 3 and 4.

[0043] Preprocessing (regularization): Input images of any size (JPG / PNG or DCM format) are first processed to uniformize pixel values. For DCM format images, pixel values ​​are also inverted. To address the issue of inconsistent image sizes, the region of interest is cropped to obtain a fixed-size image patch. For images where the location of the region of interest is uncertain, the input image is scaled to the desired size using bilinear interpolation.

[0044] Segmenting the target lesion region: The VGU-Net pre-trained segmentation model can be trained from a training dataset of the target image domain. Typically, the training image dataset requires more than 100 images, and after approximately 200 training rounds, a highly accurate pre-trained model can be obtained. After deployment, the segmented image can be derived from the regularized image through inference by the pre-trained model.

[0045] Implementation process as follows Figure 3 As shown, this technology provides a fully automated end-to-end medical image segmentation solution capable of handling images of different formats and resolutions generated by existing imaging systems. The system first normalizes the input image, extracts patches or scales the image, and processes it to the network input size. Then, a pre-trained VGU-Net model is used to segment the target lesion region on the normalized image. Figure 4 This is a schematic diagram of the input image; Figure 5 This is a schematic diagram of a regularized image; Figure 6 This is a segmentation diagram. Compared to existing methods, this invention can achieve fully automatic segmentation, and the segmentation accuracy is significantly improved compared to existing models.

[0046] Detailed steps of the method:

[0047] 1. Perform image homogenization and resizing on the input image.

[0048] 2. Obtain a training set of regularized images in the target image domain, and then perform multiple rounds of training and optimization on the VGU-Net model to obtain the model that performs best on the validation set.

[0049] 3. Use the trained VGU-Net network to predict the segmentation results of the input image.

[0050] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A medical image segmentation method based on graph neural networks and the U-Net deep learning framework, characterized in that, Includes the following steps: Step 1: Input image; Step 2: Image regularization processing; Step 3: Obtain a training set of regularized images in the target image domain, train and optimize the VGU-Net model, and obtain the model that performs best on the validation set. Step 4: Use the trained VGU-Net model to predict the segmentation result of the input image; Image homogenization and size scaling; The VGU-Net model integrates convolutional neural networks and graph neural networks; The convolutional neural network extracts image features; The graph neural network interacts with image features to transmit information; After each downsampling, the 2x2 patch of the previous level feature map is used to construct a node in the graph; Through continuous downsampling, the larger the scale of image information corresponding to the graph node, the richer the information it contains. Multi-layer downsampling allows the graph node to contain information at different scales. The self-attention mechanism of the feature map is used to calculate the similarity between nodes and to use it as the weight of the edge.

2. The medical image segmentation method based on graph neural networks and the U-Net deep learning framework as described in claim 1, characterized in that, The input images in step 1 include JPG, PNG, and DCM formats.

3. The medical image segmentation method based on graph neural networks and the U-Net deep learning framework as described in claim 1, characterized in that, Step 2 includes inverting pixel values ​​for DCM format images.

4. The medical image segmentation method based on graph neural networks and the U-Net deep learning framework as described in claim 1, characterized in that, Step 2 includes performing bilinear interpolation scaling on the image.

5. The medical image segmentation method based on graph neural networks and the U-Net deep learning framework as described in claim 1, characterized in that, The convolutional neural network uses the U-Net network framework.

6. The medical image segmentation method based on graph neural networks and the U-Net deep learning framework as described in claim 1, characterized in that, The graph neural network described uses a graph convolutional neural network.

Images