Unsupervised ventricle image segmentation method based on local comparative learning
By adopting a poolless symmetric fully convolutional encoder decoder network and comprehensive loss function in ventricular image segmentation, combined with the residual structure based on SCCON, the problem of low redundant feature extraction and segmentation accuracy in the prior art is solved, and a more efficient ventricular image segmentation effect is achieved.
Patent Information
- Application Number
- CN202410194790.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-22
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art has problems with low redundant feature extraction and segmentation accuracy in ventricular image segmentation, especially when dealing with MRI imaging plane diversity and ventricular shape heterogeneity.
Unsupervised ventricular image segmentation method based on local contrast learning is adopted, image features are extracted through a pool-free symmetric fully convolutional encoder decoder network, and a comprehensive loss function is designed to capture the microscopic and macroscopic characteristics of the ventricles. At the same time, a residual structure based on SCCON is introduced to reduce redundant features and enhance the learning of key features.
It improves the segmentation accuracy of MRI ventricular images, reduces the extraction of redundant features, enhances the model's ability to capture ventricular characteristics, and achieves more efficient image information extraction and recovery.
Smart Images
Figure CN119649018A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of image processing, and in particular to an unsupervised ventricle image segmentation method based on local contrast learning. Background Art
[0002] With the rapid development of medical imaging technology, magnetic resonance imaging (MRI) has become an important tool for studying and diagnosing neurological diseases, especially in the early diagnosis of neurodegenerative diseases such as Alzheimer's disease. In this context, the accuracy of ventricle segmentation becomes crucial, because the change in ventricle volume is a key indicator of the progression of these diseases. However, due to the high heterogeneity of ventricles in shape, size, and pixel intensity, as well as the diversity of MRI imaging planes, the automated ventricle segmentation task faces great challenges.
[0003] Traditional image segmentation methods can be roughly divided into two categories: one relies on handcrafted features, and the other is based on pre-computed and annotated atlases. Although these methods have their own strengths, they often encounter difficulties when processing complex images, and building and maintaining atlases is time-consuming and sometimes impractical.
[0004] In recent years, supervised deep learning techniques have achieved remarkable results in the field of medical image segmentation, among which U-Net and its variants have attracted much attention for achieving accurate segmentation through a large amount of expert annotations. However, these methods are overly dependent on expert annotations, which limits their widespread application.
[0005] In view of this, more and more studies have begun to explore unsupervised learning methods to reduce the reliance on expert annotations. Contrastive learning, as a self-supervised method, provides a new idea for solving the problem of limited annotations. However, most of these methods still rely on some form of supervised learning.
[0006] The prior art adopts a two-stage multi-granularity unsupervised medical image segmentation model, which provides a new solution to these problems. In the first stage, this method combines local contrast learning and local reconstruction network for feature learning, but fails to fully consider the global structure of the image. In addition, its backbone network extracts patch features through four layers of convolution, which produces a large number of redundant features. Its reconstruction network also consists of only one fully connected layer, which is not enough to effectively consider the local information of the image patch, and its multi-granularity segmentation results mainly for images have relatively low segmentation accuracy.
[0007] To this end, the present invention proposes an unsupervised ventricle image segmentation method based on local contrast learning. Summary of the invention
[0008] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an unsupervised ventricle image segmentation method based on local contrast learning.
[0009] In order to achieve the above object, the present invention adopts the following technical solutions:
[0010] An unsupervised ventricle image segmentation method based on local contrast learning comprises the following steps:
[0011] S1: Extract each pixel in the image and its surrounding local neighborhood as a patch centered on the pixel;
[0012] S2: Encode pixel-centered patches into a high-dimensional latent space;
[0013] S3: Image segmentation, using spectral k-MEANS clustering.
[0014] Preferably: the encoder comprises four convolutional layers and four transposed convolutional layers symmetrical to the four convolutional layers, the convolutional layers are used to reconstruct the entire image, and the transposed convolutional layers are used to reconstruct local pixels.
[0015] Preferably: the kernel sizes of the four convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all the same, and the output sizes of the four convolutional layers are 128*128*16, 128*128*32, 128*128*64, and 128*128*128, respectively.
[0016] Preferably: the kernel sizes of the four transposed convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all padding=1, and the output sizes of the four transposed convolutional layers are 128*128*64, 128*128*32, 128*128*16, and 128*128*3, respectively.
[0017] Preferably: a residual structure and a SCCON convolution module are arranged between the convolution layers for feature enhancement.
[0018] Preferably: the SCCON convolution module comprises a spatial reconstruction unit SRU and a channel reconstruction unit CRU placed in sequence.
[0019] Preferably: the working logic of the SCCON convolution module is:
[0020] A1: For the intermediate input feature X in the bottleneck residual block, the spatial refinement feature is first obtained through the spatial reconstruction unit SRU operation;
[0021] A2: Then use the channel reconstruction unit CRU operation to obtain channel refinement features.
[0022] Preferably, in the step S1, a loss function is used, and the loss function is: and is the embedding vector for each anchor patch, Ω + is the position of the positive sample set to the anchor point, Ω - The location of the negative sample set to the anchor point, τ is the temperature parameter, and sim(·) is the similarity function.
[0023] Preferably: the step S3 includes the following steps:
[0024] S31: Each pixel-centered patch in the input image is mapped to a 128-dimensional vector;
[0025] S32: Reduce the dimension of the data through the PHATE algorithm to obtain a low-dimensional representation that can capture the key structures and changes in the data;
[0026] S33: Apply the K-MEANS algorithm in the PHATE library to the reduced-dimensional data to perform image segmentation.
[0027] The beneficial effects of the present invention are:
[0028] 1. The present invention can accept image inputs of various sizes through a poolless symmetric fully convolutional encoder-decoder network, and efficiently extract and restore image information; at the same time, a comprehensive loss function, pixel patch contrast loss, local contrast learning loss and global reconstruction loss are designed to more effectively capture the micro and macro characteristics of the ventricles; in addition, a SCCON-based residual structure is added to the encoder and decoder, which not only removes redundant features but also enhances the learning of key features, greatly improving the segmentation accuracy of MRI ventricle images.
[0029] 2. The present invention designs a symmetric fully convolutional network based on SCCON, which is suitable for various image sizes, and introduces a residual structure based on SCCON convolution to enhance core features and reduce redundant features, further improving the model performance. At the same time, spectral K-MEANS is used for segmentation to further improve the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A flowchart of an unsupervised ventricle image segmentation method based on local contrast learning proposed by the present invention;
[0031] Figure 2 This is a network architecture diagram of an unsupervised ventricle image segmentation method based on local contrast learning proposed by the present invention;
[0032] Figure 3 This is a structural diagram of the SCCON convolution module of an unsupervised ventricle image segmentation method based on local contrast learning proposed in the present invention. DETAILED DESCRIPTION
[0033] The technical solution of the present invention is further described in detail below in conjunction with specific implementation methods.
[0034] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", and "set" should be understood in a broad sense, for example, they can be fixedly connected or set, or detachably connected or set, or integrally connected or set. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0035] Embodiment 1:
[0036] An unsupervised ventricle image segmentation method based on local contrast learning is implemented by an encoder and specifically includes the following steps:
[0037] S1: Extract each pixel in the image and its surrounding local neighborhood as a patch centered on the pixel;
[0038] S2: Encode pixel-centered patches into a high-dimensional latent space;
[0039] S3: Image segmentation, using spectral k-MEANS clustering.
[0040] The encoder includes four convolutional layers and four transposed convolutional layers symmetrical to the four convolutional layers, the convolutional layers are used to reconstruct the entire image, and the transposed convolutional layers are used to reconstruct local pixels.
[0041] The kernel sizes of the four convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all the same, and the output sizes of the four convolutional layers are 128*128*16, 128*128*32, 128*128*64, and 128*128*128, respectively.
[0042] The kernel sizes of the four transposed convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all padding=1, and the output sizes of the four transposed convolutional layers are 128*128*64, 128*128*32, 128*128*16, and 128*128*3, respectively.
[0043] A residual structure and a SCCON convolution module are arranged between the convolution layers for feature enhancement.
[0044] The SCCON convolution module includes a spatial reconstruction unit SRU and a channel reconstruction unit CRU placed in a sequential manner.
[0045] The working logic of the SCCON convolution module is:
[0046] A1: For the intermediate input feature X in the bottleneck residual block, the spatial refinement feature is first obtained through the spatial reconstruction unit SRU operation;
[0047] A2: Then use the channel reconstruction unit CRU operation to obtain channel refinement features.
[0048] The SCConv module makes full use of the spatial redundancy and channel redundancy between features and can be seamlessly integrated into any CNN architecture to reduce the redundancy between intermediate feature maps and enhance the feature representation of CNN.
[0049] In the step S1, a loss function is used to implement the loss function, and the loss function is: The loss function combines the pixel patch contrast loss, the global reconstruction loss, and the local contrastive learning loss. The goal of the contrastive loss is to ensure that the encoder maps similar pixel-centered patches to the embedding space in an organized manner. The neighbor heuristic and the image structure similarity metric (SSIM) are used to determine the positive and negative pairs. Positive examples are local neighborhood patches with a similarity to the anchor patch above a threshold, while negative examples are patches that are far from the anchor patch or fail the similarity check. In the embedding space, contrastive learning is performed to compare the embedding vectors of each anchor patch. and In this way, positive and negative pairs are generated and contrastive loss is calculated without using supervised labels, and invariance in the embedding space is established. The negative sample sampling process is simplified, and positive samples from different anchor patches are used as negative samples of the current anchor patch to improve the mining speed.
[0050] Minimizing the loss function will encourage the network to convert the positive sample set Ω + is embedded close to the anchor point, and the degree of closeness is determined by the negative set Ω - The distance between the anchor points is scaled. In this way, similar pairs are encouraged to be embedded closer and different pairs are encouraged to be embedded farther. The temperature parameter τ controls the scaling of the distance used in the contrastive loss, preferably: 0.5. sim(·) is the similarity function, preferably cosine similarity.
[0051] The pixel patch contrast loss uses the reconstructed pixels and the original image patches for loss estimation. This loss function is very effective for MRI images. It can force the network's feature vector to fully preserve the information of the pixel corresponding to the patch, which is very beneficial for the second-stage segmentation task. In addition, this reconstructed pixel point can be well matched with the introduced global loss, so that the feature space can better retain the information and structure of the original image without damaging it. Specifically, for the embedded Z ij ∈R d , pixels x ij The contrast loss with the patch is:
[0052] During the training process, in order to better capture the overall structure and texture information of the image, the reconstruction loss of the global image is also introduced. The decoder is also used to reconstruct the global image, using f global (·):R w*h*d →R w*h*c To process the global feature map output by the encoder, the output is
[0053] The total objective function of the total loss network is the sum of the contrast loss and the reconstruction loss, which are balanced by the weight coefficient λ∈[0, 1] that controls the relative size:
[0054] loss = λ·l contrastice +(1-2·λ)·l recon +λ·l global
[0055] Preferably: λ=0.01.
[0056] The step S3 includes the following steps:
[0057] S31: Each pixel-centered patch in the input image is mapped to a 128-dimensional vector;
[0058] S32: Reduce the dimension of the data through the PHATE algorithm to obtain a low-dimensional representation that can capture the key structures and changes in the data;
[0059] S33: Apply the K-MEANS algorithm in the PHATE library to the reduced-dimensional data to perform image segmentation.
[0060] The study used the publicly available magnetic resonance images (MRI) dataset of the Alzheimer's Disease Neuroimaging Initiative, and randomly selected 100 T1-weighted brain MRIs. Radiologists manually segmented the ventricles in these images. The dataset division was based on the method of the CUTS paper and is unique. Due to the use of unsupervised or self-supervised learning methods, CUTS recommends that the data be divided into a training set and a validation set in a 7:3 ratio during model training, and the entire dataset is used as a test set to evaluate the segmentation effect. This division method is considered reasonable because the model training phase does not rely on true labels. To verify the rationality of this division method, the data was retrained and tested in an 8:1:1 ratio, and it was found that the results obtained by the two methods were similar.
[0061] The specific process of the experiment is:
[0062] Experimental environment
[0063] The model is implemented in Python 3.8 and PyTorch, which is suitable for deep learning tasks. The training and testing environment includes a server with two DCU accelerator cards with 16GB video memory and 40GB memory, which provides strong hardware support for deep learning tasks. The learning rate is set to 0.001. The batch size is set to 16, which means that 16 samples are used to calculate the loss and update the weights each time the model is trained. The model is trained for 200 rounds, each of which contains one iteration of the entire training dataset to help the model gradually converge to the best performance.
[0064] Evaluation indicators
[0065] The evaluation indicators used are Dice coefficient, Hausdorff distance, SSIM, and RMSE. These indicators are used to compare different fields such as sample similarity, distance between point sets, image structure similarity, and model prediction ability. They have different applications and calculation methods in different scenarios and can be adjusted according to specific needs.
[0066] In the study, the proposed method was used to process the ventricle image dataset and compared with the currently popular advanced methods. The experimental results verify the effectiveness of the segmentation method proposed in this paper. The specific comparison data are shown in the following table. Compared with other unsupervised segmentation methods, the segmentation map generated by the proposed method has a higher similarity with the real labeled data, which can be reflected in the higher SSIM value and RMSE value. The key DICE index was improved by 8.7%.
[0067]
[0068] In order to further verify the effectiveness of the added SCCON module, under the same experimental conditions, the framework without the SCCON module was used as the baseline network, and then the SCCON module was added for ablation study. These experiments were conducted on the dataset, and the experimental results are shown in the following table, with the best data highlighted in black. By analyzing these two sets of ablation data, it can be seen that the network with the SCCON residual module has a 1.2% improvement in the DICE indicator compared to the baseline network (the network without the SCCON module), and has steadily improved in the Hausdorff distance, SSIM similarity coefficient and RMSE indicators. Overall, the combination of these two structures achieves the best overall segmentation effect. As shown in the table below, the joint application of these two structures enables the network model to reach the best level in various indicators.
[0069]
[0070] The present invention can accept image inputs of various sizes and efficiently extract and restore image information. At the same time, a comprehensive loss function is designed, including pixel patch contrast loss, local contrast learning loss and global reconstruction loss, to more effectively capture the microscopic and macroscopic characteristics of the ventricles. In addition, a SCCON-based residual structure is added to the encoder and decoder, which not only removes redundant features but also enhances the learning of key features, thereby greatly improving the segmentation accuracy of MRI ventricle images.
[0071] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. An unsupervised ventricle image segmentation method based on local contrast learning, implemented by an encoder, characterized in that: The specific steps include: S1: Extract each pixel in the image and its surrounding local neighborhood as a patch centered on the pixel; S2: Encode pixel-centered patches into a high-dimensional latent space; S3: Image segmentation, using spectral k-MEANS clustering.
2. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 1, characterized in that: The encoder includes four convolutional layers and four transposed convolutional layers symmetrical to the four convolutional layers, the convolutional layers are used to reconstruct the entire image, and the transposed convolutional layers are used to reconstruct local pixels.
3. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 2, characterized in that: The kernel sizes of the four convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all the same, and the output sizes of the four convolutional layers are 128*128*16, 128*128*32, 128*128*64, and 128*128*128, respectively.
4. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 2, characterized in that: The kernel sizes of the four transposed convolutional layers are all (5, 5), the step sizes are all 1, the padding methods are all padding=1, and the output sizes of the four transposed convolutional layers are 128*128*64, 128*128*32, 128*128*16, and 128*128*3, respectively.
5. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 2, characterized in that: A residual structure and a SCCON convolution module are arranged between the convolution layers for feature enhancement.
6. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 5, characterized in that: The SCCON convolution module includes a spatial reconstruction unit SRU and a channel reconstruction unit CRU placed in a sequential manner.
7. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 6, characterized in that: The working logic of the SCCON convolution module is: A1: For the intermediate input feature X in the bottleneck residual block, the spatial refinement feature is first obtained through the spatial reconstruction unit SRU operation; A2: Then use the channel reconstruction unit CRU operation to obtain channel refinement features.
8. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 1, characterized in that: In the step S1, a loss function is used, and the loss function is: and is the embedding vector for each anchor patch, Ω + is the position of the positive sample set to the anchor point, Ω - The location of the negative sample set to the anchor point, τ is the temperature parameter, and sim(·) is the similarity function.
9. The unsupervised ventricle image segmentation method based on local contrast learning according to claim 1, characterized in that: The step S3 includes the following steps: S31: Each pixel-centered patch in the input image is mapped to a 128-dimensional vector; S32: Reduce the dimension of the data through the PHATE algorithm to obtain a low-dimensional representation that can capture the key structures and changes in the data; S33: Apply the K-MEANS algorithm in the PHATE library to the reduced-dimensional data to perform image segmentation.