Method for improving medical image registration network performance based on comparative learning

Abstract

The invention discloses a method for improving medical image registration network performance based on comparative learning, which is characterized in that a random deformation field and a deformation field generated by a network in a training process are adopted to deform a target image, an edge point set of target image features is used as an anchor point, and an edge point set of registration image features is used as a positive sample; random non-zero point sets of the features of the two images are negative samples, and the performance of the medical image registration network is optimized by comparing and learning a loss function. Compared with the prior art, the method has the advantages that the feature extraction capability of the medical image registration model encoder is enhanced, extra resources do not need to be introduced in the reasoning stage, the final registration precision is effectively improved, and the method for improving the contrast learning encoder has the characteristics of convenience, high efficiency and plug and play, and is suitable for popularization and application. The method can be embedded into most of current medical image registration networks, so that the performance of the model on registration tasks is improved, and the problem that the feature extraction capability of an encoder in the medical image registration networks is insufficient is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of medical image registration and deep learning technology, and in particular to a method for improving the performance of a medical image registration network based on contrastive learning. Background Art

[0002] In recent years, medical image registration technology has been widely used in computer-aided diagnosis, surgical planning, and image-guided therapy. Medical image registration aims to align images of different modalities or time points through spatial transformation, thereby assisting doctors in more accurate analysis and diagnosis. Convolutional neural networks (CNNs) are widely used in medical image registration tasks due to their excellent performance in feature extraction and expression. However, traditional registration methods often rely on fixed similarity metrics (such as mean square error or mutual information), which results in insufficient encoder performance when processing images with large non-rigid deformations.

[0003] As a self-supervised learning method, contrastive learning enhances feature representation by bringing positive pairs closer and negative pairs farther apart in feature space. Contrastive learning has achieved remarkable results in natural image classification, object detection and other tasks. However, there are still many challenges in directly applying contrastive learning to medical image registration tasks, such as the need for additional training.

[0004] In summary, the existing medical image registration technology relies on fixed similarity metrics, such as mean square error or mutual information. When processing images with large non-rigid deformations, there are problems such as insufficient overall encoder performance. Summary of the invention

[0005] The purpose of the present invention is to improve the performance of the medical image registration network based on contrastive learning, which is designed to address the deficiencies of the prior art. The method uses a random deformation field and a deformation field generated by the network during the training process to deform the target image, uses the edge point set of the target image features as anchor points, the edge point set of the registration image features as positive samples, and the random non-zero point set of the two image features as negative samples, and improves the overall performance of the encoder through a contrastive learning loss function. The method effectively improves the performance of the registration network encoder without significantly increasing the complexity of the model. At the same time, the method can be used as an independent enhancement module (contrast learning module) plug-and-play, and can be flexibly embedded in different registration network architectures, thereby further improving the accuracy and stability of medical image registration, improving the encoder performance of the registration network without consuming additional resources in the reasoning stage, and optimizing the final registration result, which has broad application prospects.

[0006] The specific technical scheme for implementing the present invention is: a method for improving the performance of a volume medical image registration network based on contrastive learning, which is characterized by using a random deformation field and a deformation field generated by the network during training to deform the target image, using the edge point set of the target image features as anchor points, the edge point set of the registration image features as positive samples, and the random non-zero point set of the two image features as negative samples, and optimizing the performance of the medical image registration network by contrastive learning loss function, which specifically includes the following steps:

[0007] Step a: Randomly select edge points of the target image to obtain N edge points .

[0008] Step b: Introduce a random known deformation field, deform the image to be registered, obtain the deformed image, and then use the randomly extracted edge points to calculate the deformation field. And the deformation field displacement parameter calculation, get the N edge points of the deformation image .

[0009] Step c: Randomly select non-zero points of the target image and the deformed image to obtain K non-zero points .

[0010] Step d: Input the target image and the deformed image into the registration network encoder to obtain the edge points in the low-scale feature map of the target image The corresponding eigenvector is , edge points in the low-scale feature map of the deformed image The corresponding eigenvector is , and the non-zero points in the low-scale features of the target and deformed images The corresponding eigenvector is .

[0011] Step e: Take edge points is the anchor vector, is a positive sample vector, and the non-zero points correspond to is a negative sample vector. The contrastive learning loss function is used to jointly optimize the registration network with the loss function of the target model during training to enhance the feature extraction capability of the encoder. It is expressed by the following formula (a):

[0012] (a).

[0013] in, is a hyperparameter.

[0014] Step f: Select the deformation field generated during the training process as the displacement deformation field, and repeat steps a to e, so that the registration network captures deformation information during the contrast learning process to optimize the performance of the medical image registration network.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0016] 1) It can be used as a plug-and-play module embedded in different registration networks during the training process, which can effectively improve the encoder performance and thus improve the final registration effect;

[0017] 2) The present invention has the characteristics of convenient method, light weight, high efficiency and plug-and-play, which not only improves the feature extraction capability of the encoder, but also improves the performance of the medical image registration model in the image registration task, and can achieve better results in different network model architectures;

[0018] 3) The present invention improves the performance of the encoder through contrastive learning and adopts random deformation fields and deformation fields generated during training as displacement fields, which effectively improves the feature extraction capability of the encoder and can improve the performance of the network model without introducing additional resources in the inference stage.

[0019] 4) The present invention enhances the performance of the encoder during the training process by contrasting the learning loss function, and achieves higher registration performance of the model without consuming additional inference resources.

[0020] 5) The present invention can achieve significant performance gains without consuming additional resources in the inference phase, further promoting the exploration of the combination of contrastive learning and registration networks, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the present invention;

[0022] Figure 2 This is an example diagram of the comparative learning module;

[0023] Figure 3 Comparison of the original CorrMLP network and the embedded contrastive learning module for medical image registration. DETAILED DESCRIPTION

[0024] See also Figure 1 , the present invention specifically comprises the following steps:

[0025] Step a: Create a medical image training set;

[0026] Step b: Select random edge points of the target image as anchor points;

[0027] Step c: deform the target image by introducing a random deformation field, and obtain the corresponding positive sample points according to the deformation field parameters and the anchor point coordinates;

[0028] Step d: Select random non-zero points in the target image and the registered image as negative sample points;

[0029] Step e: Input the target image and the registered image into the registration network to obtain the feature map generated by the encoder;

[0030] Step f: Compare the feature vectors corresponding to each point in the feature map and calculate the learning loss to improve the feature extraction capability of the encoder;

[0031] Step g: Similarly, replace the random deformation field with the deformation field generated during the current training process, repeat steps bf, and further improve the performance of the medical image registration network encoder, and ultimately effectively improve the registration results.

[0032] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the following specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] Example 1

[0034] See also Figure 1 The present invention takes the medical image registration model as the target model, improves the performance of the target model encoder during the training process by selecting positive and negative samples and contrasting the learning loss function, and thus optimizes the model's results on the registration task. The method mainly includes a contrast learning module, and its specific implementation steps are as follows:

[0035] Step a: Input the image into the convolutional neural network, perform shallow feature extraction through the image encoder, and obtain the intermediate feature map.

[0036] Step b: Calculate the contrastive learning loss on the output feature map.

[0037] See also Figure 2 The contrastive learning module is mainly divided into two steps: positive and negative sample selection and contrast loss calculation. First, the target image is deformed using the random deformation field and the deformation field generated during the training process. The random edge point set of the target image is used as the anchor point, the edge points corresponding to the deformed image are the positive samples, and the non-zero point sets of the two images are used as negative samples. Then, the target image and the deformed image are input into the encoder, and the contrast loss is calculated for the generated low-scale feature map, thereby effectively improving the encoder feature extraction ability of the registration network. 1000 rounds of training were performed on the FLARE and ACDC datasets respectively, with the batch size set to 1, the training step set to 100, the initial learning rate set to 0.0001, and the ADAM optimizer used.

[0038] Step c: Update the model parameters of the medical image registration network using the gradient descent method on the training dataset.

[0039] Step d: After the model training is completed, the fixed model parameters are verified on the image registration test set.

[0040] The comparison of the registration performance improvement of the model after embedding this module and the original model in the FLARE dataset and ACDC dataset is shown in Table 1 below;

[0041] Table 1 Comparison of performance improvement of registration network

[0042]

[0043] By comparing the three original networks of NICE-Net, DualPRNET++ and CorrMLP, the network architecture embedded with this module has excellent registration performance on the FLARE and ACDC datasets, and effectively improves the registration accuracy while consuming additional inference resources. For example, the CorrMLP network embedded with CN achieved a classification accuracy of 66.38% on FLARE, which is 4.83% higher than the original network (71.21%). Compared with the original network, the present invention effectively improves the registration accuracy and achieves better results in the registration task.

[0044] See also Figure 3 Compared with the original CorrMLP network, the registration effect of the model network after adding this module has been significantly improved. It performs well on the FLARE dataset and has better registration performance.

[0045] The above embodiments are only for further explanation of the present invention, and are not intended to limit the present invention. Any equivalent implementation of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for improving the performance of medical image registration network based on contrastive learning, characterized in that: The target image is deformed using a random deformation field and a deformation field generated by the network during training. The edge point set of the target image features is used as an anchor point, the edge point set of the registration image features is used as a positive sample, and the random non-zero point set of the two image features is used as a negative sample. The performance of the medical image registration network is optimized by comparing the learning loss function. Specifically, the following steps are included: Step a: Randomly select edge points of the target image to obtain N edge points ; Step b: Introduce a random known deformation field, deform the image to be registered, obtain the deformed image, and then use the randomly extracted edge points to calculate the deformation field. And the deformation field displacement parameter calculation, get the N edge points of the deformation image ; Step c: Randomly select non-zero points of the target image and the deformed image to obtain K non-zero points ; Step d: Input the target image and the deformed image into the registration network encoder to obtain the edge points in the low-scale feature map of the target image The corresponding eigenvector is , edge points in the low-scale feature map of the deformed image The corresponding eigenvector is , and the non-zero points in the low-scale features of the target and deformed images The corresponding eigenvector is ; Step e: Take edge points is the anchor vector, is a positive sample vector, and the non-zero points correspond to is a negative sample vector. The contrastive learning loss function is used to jointly optimize the registration network with the loss function of the target model during training to enhance the feature extraction capability of the encoder. It is expressed by the following formula (a): （a）； in, is a hyperparameter; Step f: Select the deformation field generated during the training process as the displacement deformation field, repeat steps a to e, so that the registration network captures the deformation information during the contrast learning process, and obtains a medical image registration network with optimized performance.

Images