Multi-modal medical image registration method combining intelligent agent and attention mechanism

Abstract

The invention relates to a multi-modal medical image rigid registration algorithm based on deep reinforcement learning, which is a new end-to-end multi-modal image registration method, is trained by an asynchronous dominant actor commentator (A3C), can simulate the gradual registration process of human experts, and enhances the interpretability of the registration result. In view of a severe challenge of a multi-modal registration task in the aspect of calculation complexity, reinforcement learning and an attention mechanism are combined, so that an intelligent agent can capture more abstract high-dimensional features, introduction of a deep neural network with huge parameter quantity is avoided, and the characteristics of light weight and high efficiency of the network are kept, so that a model is easy to train, and the accuracy of registration is improved. And the method has strong robustness and generalization ability, and can drive the model to distort the moving image along the correct direction.

Description

technical field [0001] The invention relates to the field of medical image processing, in particular to a multimodal medical image registration method combining an agent and an attention mechanism. Background technique [0002] Medical image registration is the process of mapping images to the same coordinate system by finding the spatial correspondence between different images. Images from different modalities often contain different information. For example, computed tomography (CT) can obtain good imaging results when facing hollow organs such as bones and lungs, and magnetic resonance imaging (MRI) can obtain good soft tissue contrast. Therefore, the difference between normal tissue and diseased tissue can be effectively observed, and multimodal medical image registration can fuse information from different modalities to assist doctors in accurately diagnosing a patient's condition. [0003] Methods of medical image registration now generally include traditional feature...

AI Technical Summary

Problems solved by technology

The method based on Deep Reinforcement Learning (DRL) regards the registration process as a Markov Decision Process (MDP), allowing the DRL agent to freely explore in the predefined action space and accumulate in the process of trial and error. According to experience, in the end, corresponding decisions can be made quickly according to the specific environment, and high-precision registration can be completed, which enables each step of image registration to be visualized and presented, overcoming the opacity of the deep learning method. However, The huge computational overhead of deep neural networks will make it difficult for the model to converge

The process of free exploration enables the agent to obtain stronger generalization performance, but using the experience replay pool to store state-action pairs requires a lot of expensive memory space

[0006] Most of the current deep learning methods have poor interpretability for registration results, and the extraction of high-dimensional abstract features usually requires a network with deeper layers and more complex structures, which puts forward higher requirements for computing performance. Most of the deep reinforcement learning methods A huge memory space is required to save historical experience, how to simplify the registration model and improve the registration efficiency has become an urgent problem to be solved in medical image registration

Images